[K3MNIST2] - Simple classification with CNNĀ¶
An example of classification using a convolutional neural network for the famous MNIST datasetObjectives :Ā¶
- Recognizing handwritten numbers
- Understanding the principle of a classifier DNN network
- Implementation with Keras
The MNIST dataset (Modified National Institute of Standards and Technology) is a must for Deep Learning.
It consists of 60,000 small images of handwritten numbers for learning and 10,000 for testing.
What we're going to do :Ā¶
- Retrieve data
- Preparing the data
- Create a model
- Train the model
- Evaluate the result
Step 1 - Init python stuffĀ¶
InĀ [1]:
import os
os.environ['KERAS_BACKEND'] = 'torch'
import keras
import numpy as np
import matplotlib.pyplot as plt
import sys,os
from importlib import reload
# Init Fidle environment
import fidle
run_id, run_dir, datasets_dir = fidle.init('K3MNIST2')
FIDLE - Environment initialization
Version : 2.3.2 Run id : K3MNIST2 Run dir : ./run/K3MNIST2 Datasets dir : /lustre/fswork/projects/rech/mlh/uja62cb/fidle-project/datasets-fidle Start time : 22/12/24 21:21:30 Hostname : r3i6n0 (Linux) Tensorflow log level : Info + Warning + Error (=0) Update keras cache : False Update torch cache : False Save figs : ./run/K3MNIST2/figs (True) keras : 3.7.0 numpy : 2.1.2 sklearn : 1.5.2 yaml : 6.0.2 matplotlib : 3.9.2 pandas : 2.2.3 torch : 2.5.0
Verbosity during training : 0 = silent, 1 = progress bar, 2 = one line per epoch
InĀ [2]:
fit_verbosity = 1
Override parameters (batch mode) - Just forget this cell
InĀ [3]:
fidle.override('fit_verbosity')
** Overrided parameters : ** fit_verbosity : 2
Step 2 - Retrieve dataĀ¶
MNIST is one of the most famous historic dataset.
Include in Keras datasets
InĀ [4]:
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(-1,28,28,1)
x_test = x_test.reshape(-1,28,28,1)
print("x_train : ",x_train.shape)
print("y_train : ",y_train.shape)
print("x_test : ",x_test.shape)
print("y_test : ",y_test.shape)
x_train : (60000, 28, 28, 1) y_train : (60000,) x_test : (10000, 28, 28, 1) y_test : (10000,)
Step 3 - Preparing the dataĀ¶
InĀ [5]:
print('Before normalization : Min={}, max={}'.format(x_train.min(),x_train.max()))
xmax=x_train.max()
x_train = x_train / xmax
x_test = x_test / xmax
print('After normalization : Min={}, max={}'.format(x_train.min(),x_train.max()))
Before normalization : Min=0, max=255
After normalization : Min=0.0, max=1.0
Have a lookĀ¶
InĀ [6]:
fidle.scrawler.images(x_train, y_train, [27], x_size=5,y_size=5, colorbar=True, save_as='01-one-digit')
fidle.scrawler.images(x_train, y_train, range(5,41), columns=12, save_as='02-many-digits')
Saved: ./run/K3MNIST2/figs/01-one-digit
Saved: ./run/K3MNIST2/figs/02-many-digits
InĀ [7]:
model = keras.models.Sequential()
model.add( keras.layers.Input((28,28,1)) )
model.add( keras.layers.Conv2D(8, (3,3), activation='relu') )
model.add( keras.layers.MaxPooling2D((2,2)))
model.add( keras.layers.Dropout(0.2))
model.add( keras.layers.Conv2D(16, (3,3), activation='relu') )
model.add( keras.layers.MaxPooling2D((2,2)))
model.add( keras.layers.Dropout(0.2))
model.add( keras.layers.Flatten())
model.add( keras.layers.Dense(100, activation='relu'))
model.add( keras.layers.Dropout(0.5))
model.add( keras.layers.Dense(10, activation='softmax'))
InĀ [8]:
model.summary()
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
Model: "sequential"
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā³āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā³āāāāāāāāāāāāāāāāāā ā Layer (type) ā Output Shape ā Param # ā ā”āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā© ā conv2d (Conv2D) ā (None, 26, 26, 8) ā 80 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāā¤ ā max_pooling2d (MaxPooling2D) ā (None, 13, 13, 8) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāā¤ ā dropout (Dropout) ā (None, 13, 13, 8) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāā¤ ā conv2d_1 (Conv2D) ā (None, 11, 11, 16) ā 1,168 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāā¤ ā max_pooling2d_1 (MaxPooling2D) ā (None, 5, 5, 16) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāā¤ ā dropout_1 (Dropout) ā (None, 5, 5, 16) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāā¤ ā flatten (Flatten) ā (None, 400) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāā¤ ā dense (Dense) ā (None, 100) ā 40,100 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāā¤ ā dropout_2 (Dropout) ā (None, 100) ā 0 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāā¤ ā dense_1 (Dense) ā (None, 10) ā 1,010 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā“āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā“āāāāāāāāāāāāāāāāāā
Total params: 42,358 (165.46 KB)
Trainable params: 42,358 (165.46 KB)
Non-trainable params: 0 (0.00 B)
Step 5 - Train the modelĀ¶
InĀ [9]:
batch_size = 512
epochs = 16
history = model.fit( x_train, y_train,
batch_size = batch_size,
epochs = epochs,
verbose = fit_verbosity,
validation_data = (x_test, y_test))
Epoch 1/16
118/118 - 3s - 25ms/step - accuracy: 0.6161 - loss: 1.1591 - val_accuracy: 0.9198 - val_loss: 0.3023
Epoch 2/16
118/118 - 1s - 12ms/step - accuracy: 0.8738 - loss: 0.4122 - val_accuracy: 0.9485 - val_loss: 0.1699
Epoch 3/16
118/118 - 1s - 12ms/step - accuracy: 0.9104 - loss: 0.2956 - val_accuracy: 0.9632 - val_loss: 0.1253
Epoch 4/16
118/118 - 1s - 12ms/step - accuracy: 0.9283 - loss: 0.2369 - val_accuracy: 0.9672 - val_loss: 0.1057
Epoch 5/16
118/118 - 1s - 12ms/step - accuracy: 0.9361 - loss: 0.2096 - val_accuracy: 0.9723 - val_loss: 0.0885
Epoch 6/16
118/118 - 1s - 11ms/step - accuracy: 0.9434 - loss: 0.1871 - val_accuracy: 0.9752 - val_loss: 0.0775
Epoch 7/16
118/118 - 1s - 12ms/step - accuracy: 0.9496 - loss: 0.1689 - val_accuracy: 0.9782 - val_loss: 0.0695
Epoch 8/16
118/118 - 1s - 12ms/step - accuracy: 0.9506 - loss: 0.1592 - val_accuracy: 0.9793 - val_loss: 0.0641
Epoch 9/16
118/118 - 1s - 12ms/step - accuracy: 0.9559 - loss: 0.1477 - val_accuracy: 0.9812 - val_loss: 0.0593
Epoch 10/16
118/118 - 1s - 12ms/step - accuracy: 0.9577 - loss: 0.1395 - val_accuracy: 0.9820 - val_loss: 0.0558
Epoch 11/16
118/118 - 1s - 12ms/step - accuracy: 0.9600 - loss: 0.1330 - val_accuracy: 0.9827 - val_loss: 0.0545
Epoch 12/16
118/118 - 1s - 11ms/step - accuracy: 0.9614 - loss: 0.1273 - val_accuracy: 0.9836 - val_loss: 0.0507
Epoch 13/16
118/118 - 1s - 12ms/step - accuracy: 0.9621 - loss: 0.1226 - val_accuracy: 0.9844 - val_loss: 0.0483
Epoch 14/16
118/118 - 1s - 12ms/step - accuracy: 0.9654 - loss: 0.1150 - val_accuracy: 0.9836 - val_loss: 0.0475
Epoch 15/16
118/118 - 1s - 12ms/step - accuracy: 0.9655 - loss: 0.1147 - val_accuracy: 0.9850 - val_loss: 0.0461
Epoch 16/16
118/118 - 1s - 12ms/step - accuracy: 0.9674 - loss: 0.1110 - val_accuracy: 0.9859 - val_loss: 0.0433
InĀ [10]:
score = model.evaluate(x_test, y_test, verbose=0)
print(f'Test loss : {score[0]:4.4f}')
print(f'Test accuracy : {score[1]:4.4f}')
Test loss : 0.0433 Test accuracy : 0.9859
6.2 - Plot historyĀ¶
InĀ [11]:
fidle.scrawler.history(history, figsize=(6,4), save_as='03-history')
Saved: ./run/K3MNIST2/figs/03-history_0
Saved: ./run/K3MNIST2/figs/03-history_1
6.3 - Plot resultsĀ¶
InĀ [12]:
#y_pred = model.predict_classes(x_test) Deprecated after 01/01/2021 !!
y_sigmoid = model.predict(x_test, verbose=fit_verbosity)
y_pred = np.argmax(y_sigmoid, axis=-1)
fidle.scrawler.images(x_test, y_test, range(0,200), columns=12, x_size=1, y_size=1, y_pred=y_pred, save_as='04-predictions')
313/313 - 1s - 3ms/step
Saved: ./run/K3MNIST2/figs/04-predictions
6.4 - Plot some errorsĀ¶
InĀ [13]:
errors=[ i for i in range(len(x_test)) if y_pred[i]!=y_test[i] ]
errors=errors[:min(24,len(errors))]
fidle.scrawler.images(x_test, y_test, errors[:15], columns=6, x_size=2, y_size=2, y_pred=y_pred, save_as='05-some-errors')
Saved: ./run/K3MNIST2/figs/05-some-errors
InĀ [14]:
fidle.scrawler.confusion_matrix(y_test,y_pred,range(10),normalize=True, save_as='06-confusion-matrix')
Saved: ./run/K3MNIST2/figs/06-confusion-matrix
InĀ [15]:
fidle.end()
End time : 22/12/24 21:22:24
Duration : 00:00:54 817ms
This notebook ends here :-)
https://fidle.cnrs.fr
A few things you can do for fun:
- Changing the network architecture (layers, number of neurons, etc.)
- Display a summary of the network
- Retrieve and display the softmax output of the network, to evaluate its "doubts".
