Keras binary classifier
After exploring the dataset, it is time to build our classifier so that we can recognize a state of heart disease from the results of clinical tests. We prepare the data before proceeding. We split the starting data into two sets—a training set and a test set. The training set is used to train a classification model, and the test set to used to test model performance.
To split the data, the scikit-learn library has been used—more specifically, the sklearn.model_selection.train_test_split() function has been used. This function quickly computes a random split into training and test sets. Let's start by importing the function:
from sklearn.model_selection import train_test_split
Now, we have to split the two DataFrames:
Input_train, Input_test, Target_train, Target_test = train_test_split(InputScaled, Target, test_size = 0.30, random_state = 5)
print(Input_train.shape)
print(Input_test.shape)
print(Target_train.shape)
print(Target_test.shape)
In the train_test_split() function, four parameters are passed: InputScaled, Target, test_size, and random_state. The InputScaled and Target parameters are inputs and target DataFrames. The test_size parameters could take the following types—float, int, or None, optional (default=0.25). If it is a float, the value should be between 0.0 and 1.0 and should represent the proportion of the dataset to include in the test split. If it is an int, the value represents the absolute number of test samples. If it is None, the value is set to the complement of the train size. By default, the value is set to 0.25. In our case, we set test_size = 0.30, which means that 30% of the data is divided up as test data. Finally, the random_state parameter is used to set the seed used by the random number generator. In this way, the repeatability of the splitting operation is guaranteed. The following results are returned:
(207, 13)
(89, 13)
(207, 1)
(89, 1)
So, the starting InputScaled DataFrame is splitted in two dataset having 207 rows (Input_train) and 89 rows (Input_test). A similar subdivision was made for the Target.
To perform a neural network classification analysis, we will use the Keras Sequential model. As we said in Chapter 1, Getting Started with Keras, to create a Keras Sequential model, we must follow these steps:
- Import the Sequential class from keras.models
- Stack the layers using the .add() method
- Configure the learning process using the .compile() method
- Train the model on the train dataset using the .fit() method
First, we have to load the libraries that are needed to run the analysis:
from keras.models import Sequential
from keras.layers import Dense
Three layer classes have been imported: Sequential, Dense, and metrics. The Sequential class is used to define a linear stack of network layers that make up a model. In the following code, we will use the Sequential constructor to create the model, which will then be enriched with layers by using the add() method. The Dense class is used to instantiate a Dense layer, which is the basic feedforward fully connected layer. Let's move on to building the model:
model = Sequential()
model.add(Dense(30, input_dim=13, activation='tanh'))
model.add(Dense(20, activation='tanh'))
model.add(Dense(1, activation='sigmoid'))
In the first row, we have set the type of the model as Sequential. Then, we have added the layers. We have already said that we will use a completely connected network structure with three levels. Fully connected levels are defined using the Dense class. The first is the most important because it is the level that defines the input—it must have the right number of inputs. This can be specified with the input_dim argument and set to 13 for the 13 input columns. We passed three parameters—30, input_dim = 13, and activation = ' tanh '. 30 (units) is a positive integer representing the dimensionality of the output space, which denotes the number of neurons in the level. Finally, activation='tanh' is used to set the activation function (hyperbolic tangent activation function). The second layer has 20 neurons and the tanh activation function. Finally, the output layer has a single neuron (output) and the sigmoid activation function. Before training a model, you need to configure the learning process, which is done via the compile() method:
model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])
Three arguments are passed:
- The adam optimizer: An algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments
- The binary_crossentropy loss function: We have used logarithmic loss, which for a binary classification problem is defined in Keras as binary_crossentropy
- The accuracy metric: A metric is a function that is used to evaluate the performance of your model during training and testing
To train the model, the fit() method is used, as follows:
model.fit(Input_train, Target_train, epochs=1000, verbose=1)
Four arguments are passed:
- Input_train: Array of input training data.
- Target_train: Array of target (label) data.
- epochs=1000: Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.
- verbose=1: An integer, either 0, 1, or 2. Verbosity mode: 0 = silent, 1 = progress bar, 2 = one line per epoch.
When the fit() function is used, the loss and the accuracy at the end of each training epoch is displayed, as shown in the following results:
To print a summary of the model, simply type the following:
model.summary()
The following results are given:
Layer (type) Output Shape Param #
=================================================================
dense_28 (Dense) (None, 30) 420
_________________________________________________________________
dense_29 (Dense) (None, 20) 620
_________________________________________________________________
dense_30 (Dense) (None, 1) 21
=================================================================
Total params: 1,061
Trainable params: 1,061
Non-trainable params: 0
To evaluate the performance of the model we have just adapted, we must use the evaluate() function, as follows:
score = model.evaluate(Input_test, Target_test, verbose=0)
print('Keras Model Accuracy = ',score[1])
This function returns the loss value and metrics values for the model in test mode. Computation is done in batches. The following result is returned:
Keras Model Accuracy = 0.8089887673935193
To verify the model's capacity in classifying heart disease as a function of the thirteen variables assumed, we can display the predicted values against the current ones. To do this, we first perform the prediction on all the observations contained in the test dataset. We can use the predict() function for this, as follows:
Target_Classification = model.predict(Input_test)
Target_Classification = (Target_Classification > 0.5)
The second line has been added to obtain a binary result. Now, we can compare the results of the classification with the actual values. The best way to do this is to use a confusion matrix. In a confusion matrix, our classification results are compared with real data.
The strength of a confusion matrix is that it identifies the nature of the classification errors, as well as their quantities. In this matrix, the diagonal cells show the number of cases that were correctly classified; all the other cells show the misclassified cases. To calculate the confusion matrix, we can use the confusion_matrix() function that's contained in the sklearn.metrics package. Let's start by importing the function:
from sklearn.metrics import confusion_matrix
Then, we can use this function to print the confusion matrix:
print(confusion_matrix(Target_test, Target_Classification))
The following results are returned:
[[34 8]
[ 9 38]]
72 observations of 89 were correctly classified by making 17 errors with an accuracy equal to 0.8089887673935193.