Neural network Keras model
To perform a neural network regression analysis, we will use the Keras Sequential model. As we mentioned in Chapter 1, Getting Started with Keras, to create a Keras Sequential model, we must follow these steps:
Import the Sequential class from keras.models.
Stack the layers using .add() method.
Configure the learning process using .compile() method.
Train the model on the train dataset using .fit() method.
We have omitted the import data step because our data is already available in the Python environment. We will use the following steps to perform the analysis:
First, we have to load the libraries needed to run the analysis:
from keras.models import Sequential
from keras.layers import Dense
from keras import metrics
Three layer classes have been imported, namely Sequential, Dense, and metrics:
- Sequential class: This is used to define a linear stack of network layers that make up a model. In the following code block, we will use the Sequential constructor to create the model, which will then be enriched with layers using the add () method.
- Dense class: This is used to instantiate a Dense layer, which is the basic feedforward fully connected layer. Finally, the metrics class is imported.
- Metric class: This is a function that is used to evaluate the performance of your model. Metric functions are to be supplied in the metrics parameter when a model is compiled.
Now, let's move on to building the model:
model = Sequential()
model.add(Dense(20, input_dim=13, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1, activation='linear'))
From the preceding code block, for the first row, we have set the type of the model to Sequential(). Then, we have added the layers. We have already said that we will use a completely connected network structure with three levels. Fully connected levels are defined using the Dense class.
The first is the most important because it is the level that defines the input, and it must have the right number of inputs. This can be specified with the input_dim argument and set to 13 for the 13 predictors. We passed three parameters: 20, input_dim = 13, and activation = 'relu'. 20 (units) is a positive integer representing the dimensionality of the output space, and denotes the number of neurons in the level. The input_dim = 13 attribute is the number of the input variables. Finally, the activation = 'relu' action is used to set the activation function. The second layer has 10 neurons and a relu activation function. Finally, the output layer has a single neuron (output) and a linear activation function.
Before training a model, you need to configure the learning process, which is done via the compile() method:
model.compile(optimizer='adam',loss='mean_squared_error',
metrics=['accuracy'])
In the preceding code block, three arguments are passed:
- The adam optimizer: This is an algorithm for the first-order, gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower order moments.
- The mean_squared_error loss function: The mean squared error (MSE) measures the average of the squares of the errors, that is, the average squared difference between the estimated values and what is estimated. MSE is a measure of the quality of an estimator. It is always nonnegative and values that are closer to zero are better.
- The accuracy metric: The accuracy metric is a function that is used to evaluate the performance of your model during training and testing.
To train the model, the fit() method is used, as follows:
model.fit(X_train, Y_train, epochs=1000, verbose=1)
In the preceding code block, four arguments are passed:
- X_train: Array of predictors training data.
- Y_train: Array of target (label) data.
- epochs=1000: Number of epochs to train the model. An epoch is an iteration over the entire x and y data provided.
- verbose: verbose=1: This is an integer: 0, 1, or 2. For verbosity mode, 0 = silent, 1 = progress bar, and 2 = one line per epoch.
When the fit() function is performed, the loss and the accuracy at the end of each training epoch are displayed, as shown in the following output:
Epoch 990/1000
354/354 [===] - 0s 56us/step - loss: 0.0021 - acc: 0.0282
Epoch 991/1000
354/354 [===] - 0s 28us/step - loss: 0.0021 - acc: 0.0282
Epoch 992/1000
354/354 [===] - 0s 27us/step - loss: 0.0020 - acc: 0.0282
Epoch 993/1000
354/354 [===] - 0s 35us/step - loss: 0.0021 - acc: 0.0282
Epoch 994/1000
354/354 [===] - 0s 32us/step - loss: 0.0021 - acc: 0.0282
Epoch 995/1000
354/354 [===] - 0s 23us/step - loss: 0.0020 - acc: 0.0282
Epoch 996/1000
354/354 [===] - 0s 56us/step - loss: 0.0020 - acc: 0.0282
Epoch 997/1000
354/354 [===] - 0s 28us/step - loss: 0.0023 - acc: 0.0282
Epoch 998/1000
354/354 [===] - 0s 28us/step - loss: 0.0020 - acc: 0.0282
Epoch 999/1000
354/354 [===] - 0s 34us/step - loss: 0.0022 - acc: 0.0282
Epoch 1000/1000
354/354 [===] - 0s 38us/step - loss: 0.0020 - acc: 0.0282
To print a summary of the model, we will type the following command:
model.summary()
In the following code block, we can see the results:
_______________________________________________________________
Layer (type) Output Shape Param #
===============================================================
dense_1 (Dense) (None, 20) 280
_______________________________________________________________
dense_2 (Dense) (None, 10) 210
_______________________________________________________________
dense_3 (Dense) (None, 1) 11
===============================================================
Total params: 501
Trainable params: 501
Non-trainable params: 0
Here, we can clearly see the output shape and the number of weights in each layer. To test the model's capacity for predicting the median value of owner-occupied homes as a function of the thirteen variables assumed, we can display the predicted values against the current ones.
To do this, we must first perform the prediction on all the observations contained in the test dataset. We can use the predict() function for this.
This is a generic function for predictions from the results of various Keras models:
Y_predKM = model.predict(X_test)
For the time being, we don't need to add anything else. This data will be useful to us later to make a comparison with another model.
To evaluate the performance of the model we have just adapted, we must use the evaluate() function:
score = model.evaluate(X_test, Y_test, verbose=0)
print('Keras Model')
print(score[0])
This function returns the loss value and metrics values for the model in test mode. Computation is done in batches. The following result is returned:
Keras Model
0.0038815933421901066
At first glance, it seems to be a good result, but to have confirmation of this, it is necessary to compare these results with those deriving from the application of another model. As we are doing a regression analysis, this model seems to be the most suitable one.