[K3MNIST2] - Simple classification with CNNĀ¶
An example of classification using a convolutional neural network for the famous MNIST datasetObjectives :Ā¶
- Recognizing handwritten numbers
- Understanding the principle of a classifier DNN network
- Implementation with Keras
The MNIST dataset (Modified National Institute of Standards and Technology) is a must for Deep Learning.
It consists of 60,000 small images of handwritten numbers for learning and 10,000 for testing.
What we're going to do :Ā¶
- Retrieve data
- Preparing the data
- Create a model
- Train the model
- Evaluate the result
Step 1 - Init python stuffĀ¶
InĀ [1]:
import os
os.environ['KERAS_BACKEND'] = 'torch'
import keras
import numpy as np
import matplotlib.pyplot as plt
import sys,os
from importlib import reload
# Init Fidle environment
import fidle
run_id, run_dir, datasets_dir = fidle.init('K3MNIST2')
FIDLE - Environment initialization
Version : 2.3.0 Run id : K3MNIST2 Run dir : ./run/K3MNIST2 Datasets dir : /gpfswork/rech/mlh/uja62cb/fidle-project/datasets-fidle Start time : 03/03/24 21:04:08 Hostname : r3i7n8 (Linux) Tensorflow log level : Warning + Error (=1) Update keras cache : False Update torch cache : False Save figs : ./run/K3MNIST2/figs (True) keras : 3.0.4 numpy : 1.24.4 sklearn : 1.3.2 yaml : 6.0.1 matplotlib : 3.8.2 pandas : 2.1.3 torch : 2.1.1
Verbosity during training : 0 = silent, 1 = progress bar, 2 = one line per epoch
InĀ [2]:
fit_verbosity = 1
Override parameters (batch mode) - Just forget this cell
InĀ [3]:
fidle.override('fit_verbosity')
** Overrided parameters : ** fit_verbosity : 2
Step 2 - Retrieve dataĀ¶
MNIST is one of the most famous historic dataset.
Include in Keras datasets
InĀ [4]:
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(-1,28,28,1)
x_test = x_test.reshape(-1,28,28,1)
print("x_train : ",x_train.shape)
print("y_train : ",y_train.shape)
print("x_test : ",x_test.shape)
print("y_test : ",y_test.shape)
x_train : (60000, 28, 28, 1) y_train : (60000,) x_test : (10000, 28, 28, 1) y_test : (10000,)
Step 3 - Preparing the dataĀ¶
InĀ [5]:
print('Before normalization : Min={}, max={}'.format(x_train.min(),x_train.max()))
xmax=x_train.max()
x_train = x_train / xmax
x_test = x_test / xmax
print('After normalization : Min={}, max={}'.format(x_train.min(),x_train.max()))
Before normalization : Min=0, max=255 After normalization : Min=0.0, max=1.0
Have a lookĀ¶
InĀ [6]:
fidle.scrawler.images(x_train, y_train, [27], x_size=5,y_size=5, colorbar=True, save_as='01-one-digit')
fidle.scrawler.images(x_train, y_train, range(5,41), columns=12, save_as='02-many-digits')
Saved: ./run/K3MNIST2/figs/01-one-digit
Saved: ./run/K3MNIST2/figs/02-many-digits
InĀ [7]:
model = keras.models.Sequential()
model.add( keras.layers.Input((28,28,1)) )
model.add( keras.layers.Conv2D(8, (3,3), activation='relu') )
model.add( keras.layers.MaxPooling2D((2,2)))
model.add( keras.layers.Dropout(0.2))
model.add( keras.layers.Conv2D(16, (3,3), activation='relu') )
model.add( keras.layers.MaxPooling2D((2,2)))
model.add( keras.layers.Dropout(0.2))
model.add( keras.layers.Flatten())
model.add( keras.layers.Dense(100, activation='relu'))
model.add( keras.layers.Dropout(0.5))
model.add( keras.layers.Dense(10, activation='softmax'))
InĀ [8]:
model.summary()
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
Model: "sequential"
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā³āāāāāāāāāāāāāāāāāāāāāāāāāāāā³āāāāāāāāāāāāā ā Layer (type) ā Output Shape ā Param # ā ā”āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā© ā conv2d (Conv2D) ā (None, 26, 26, 8) ā 80 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāā¤ ā max_pooling2d (MaxPooling2D) ā (None, 13, 13, 8) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāā¤ ā dropout (Dropout) ā (None, 13, 13, 8) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāā¤ ā conv2d_1 (Conv2D) ā (None, 11, 11, 16) ā 1,168 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāā¤ ā max_pooling2d_1 (MaxPooling2D) ā (None, 5, 5, 16) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāā¤ ā dropout_1 (Dropout) ā (None, 5, 5, 16) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāā¤ ā flatten (Flatten) ā (None, 400) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāā¤ ā dense (Dense) ā (None, 100) ā 40,100 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāā¤ ā dropout_2 (Dropout) ā (None, 100) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāā¤ ā dense_1 (Dense) ā (None, 10) ā 1,010 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā“āāāāāāāāāāāāāāāāāāāāāāāāāāāā“āāāāāāāāāāāāā
Total params: 42,358 (165.46 KB)
Trainable params: 42,358 (165.46 KB)
Non-trainable params: 0 (0.00 B)
Step 5 - Train the modelĀ¶
InĀ [9]:
batch_size = 512
epochs = 16
history = model.fit( x_train, y_train,
batch_size = batch_size,
epochs = epochs,
verbose = fit_verbosity,
validation_data = (x_test, y_test))
Epoch 1/16 118/118 - 11s - 90ms/step - accuracy: 0.6361 - loss: 1.0987 - val_accuracy: 0.9252 - val_loss: 0.2950 Epoch 2/16 118/118 - 9s - 76ms/step - accuracy: 0.8729 - loss: 0.4057 - val_accuracy: 0.9548 - val_loss: 0.1628 Epoch 3/16 118/118 - 9s - 77ms/step - accuracy: 0.9112 - loss: 0.2883 - val_accuracy: 0.9643 - val_loss: 0.1204 Epoch 4/16 118/118 - 9s - 76ms/step - accuracy: 0.9265 - loss: 0.2406 - val_accuracy: 0.9710 - val_loss: 0.0974 Epoch 5/16 118/118 - 9s - 76ms/step - accuracy: 0.9372 - loss: 0.2055 - val_accuracy: 0.9732 - val_loss: 0.0842 Epoch 6/16 118/118 - 9s - 76ms/step - accuracy: 0.9440 - loss: 0.1847 - val_accuracy: 0.9756 - val_loss: 0.0730 Epoch 7/16 118/118 - 9s - 77ms/step - accuracy: 0.9501 - loss: 0.1665 - val_accuracy: 0.9792 - val_loss: 0.0672 Epoch 8/16 118/118 - 9s - 76ms/step - accuracy: 0.9534 - loss: 0.1565 - val_accuracy: 0.9818 - val_loss: 0.0613 Epoch 9/16 118/118 - 9s - 76ms/step - accuracy: 0.9552 - loss: 0.1462 - val_accuracy: 0.9823 - val_loss: 0.0559 Epoch 10/16 118/118 - 9s - 77ms/step - accuracy: 0.9587 - loss: 0.1354 - val_accuracy: 0.9830 - val_loss: 0.0541 Epoch 11/16 118/118 - 9s - 76ms/step - accuracy: 0.9600 - loss: 0.1313 - val_accuracy: 0.9836 - val_loss: 0.0495 Epoch 12/16 118/118 - 9s - 76ms/step - accuracy: 0.9633 - loss: 0.1228 - val_accuracy: 0.9849 - val_loss: 0.0476 Epoch 13/16 118/118 - 9s - 77ms/step - accuracy: 0.9651 - loss: 0.1170 - val_accuracy: 0.9857 - val_loss: 0.0453 Epoch 14/16 118/118 - 9s - 76ms/step - accuracy: 0.9655 - loss: 0.1135 - val_accuracy: 0.9857 - val_loss: 0.0442 Epoch 15/16 118/118 - 9s - 76ms/step - accuracy: 0.9673 - loss: 0.1077 - val_accuracy: 0.9866 - val_loss: 0.0409 Epoch 16/16 118/118 - 9s - 77ms/step - accuracy: 0.9687 - loss: 0.1056 - val_accuracy: 0.9874 - val_loss: 0.0395
InĀ [10]:
score = model.evaluate(x_test, y_test, verbose=0)
print(f'Test loss : {score[0]:4.4f}')
print(f'Test accuracy : {score[1]:4.4f}')
Test loss : 0.0388 Test accuracy : 0.9874
6.2 - Plot historyĀ¶
InĀ [11]:
fidle.scrawler.history(history, figsize=(6,4), save_as='03-history')
Saved: ./run/K3MNIST2/figs/03-history_0
Saved: ./run/K3MNIST2/figs/03-history_1
6.3 - Plot resultsĀ¶
InĀ [12]:
#y_pred = model.predict_classes(x_test) Deprecated after 01/01/2021 !!
y_sigmoid = model.predict(x_test, verbose=fit_verbosity)
y_pred = np.argmax(y_sigmoid, axis=-1)
fidle.scrawler.images(x_test, y_test, range(0,200), columns=12, x_size=1, y_size=1, y_pred=y_pred, save_as='04-predictions')
313/313 - 1s - 4ms/step
Saved: ./run/K3MNIST2/figs/04-predictions
6.4 - Plot some errorsĀ¶
InĀ [13]:
errors=[ i for i in range(len(x_test)) if y_pred[i]!=y_test[i] ]
errors=errors[:min(24,len(errors))]
fidle.scrawler.images(x_test, y_test, errors[:15], columns=6, x_size=2, y_size=2, y_pred=y_pred, save_as='05-some-errors')
Saved: ./run/K3MNIST2/figs/05-some-errors
InĀ [14]:
fidle.scrawler.confusion_matrix(y_test,y_pred,range(10),normalize=True, save_as='06-confusion-matrix')
Saved: ./run/K3MNIST2/figs/06-confusion-matrix
InĀ [15]:
fidle.end()
End time : 03/03/24 21:07:07
Duration : 00:02:60 814ms
This notebook ends here :-)
https://fidle.cnrs.fr
A few things you can do for fun:
- Changing the network architecture (layers, number of neurons, etc.)
- Display a summary of the network
- Retrieve and display the softmax output of the network, to evaluate its "doubts".
