[K3MNIST1] - Simple classification with DNN

An example of classification using a dense neural network for the famous MNIST dataset

Objectives :

• Recognizing handwritten numbers
• Understanding the principle of a classifier DNN network
• Implementation with Keras

The MNIST dataset (Modified National Institute of Standards and Technology) is a must for Deep Learning.
It consists of 60,000 small images of handwritten numbers for learning and 10,000 for testing.

What we're going to do :

• Retrieve data
• Preparing the data
• Create a model
• Train the model
• Evaluate the result

Step 1 - Init python stuff

import os
os.environ['KERAS_BACKEND'] = 'torch'

import keras

import numpy as np
import matplotlib.pyplot as plt
import sys,os
from importlib import reload

# Init Fidle environment
import fidle

run_id, run_dir, datasets_dir = fidle.init('K3MNIST1')

FIDLE - Environment initialization

Version              : 2.3.2
Run id               : K3MNIST1
Run dir              : ./run/K3MNIST1
Datasets dir         : /lustre/fswork/projects/rech/mlh/uja62cb/fidle-project/datasets-fidle
Start time           : 22/12/24 21:21:28
Hostname             : r3i5n3 (Linux)
Tensorflow log level : Info + Warning + Error  (=0)
Update keras cache   : False
Update torch cache   : False
Save figs            : ./run/K3MNIST1/figs (True)
keras                : 3.7.0
numpy                : 2.1.2
sklearn              : 1.5.2
yaml                 : 6.0.2
matplotlib           : 3.9.2
pandas               : 2.2.3
torch                : 2.5.0

Verbosity during training : 0 = silent, 1 = progress bar, 2 = one line per epoch

fit_verbosity = 1

Override parameters (batch mode) - Just forget this cell

fidle.override('fit_verbosity')

** Overrided parameters : **
fit_verbosity        : 2

Step 2 - Retrieve data

MNIST is one of the most famous historic dataset.
Include in Keras datasets

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

print("x_train : ",x_train.shape)
print("y_train : ",y_train.shape)
print("x_test  : ",x_test.shape)
print("y_test  : ",y_test.shape)

x_train :  (60000, 28, 28)
y_train :  (60000,)
x_test  :  (10000, 28, 28)
y_test  :  (10000,)

Step 3 - Preparing the data

print('Before normalization : Min={}, max={}'.format(x_train.min(),x_train.max()))

xmax=x_train.max()
x_train = x_train / xmax
x_test  = x_test  / xmax

print('After normalization  : Min={}, max={}'.format(x_train.min(),x_train.max()))

Before normalization : Min=0, max=255

After normalization  : Min=0.0, max=1.0

Have a look

fidle.scrawler.images(x_train, y_train, [27],  x_size=5,y_size=5, colorbar=True, save_as='01-one-digit')
fidle.scrawler.images(x_train, y_train, range(5,41), columns=12, save_as='02-many-digits')

Saved: ./run/K3MNIST1/figs/01-one-digit

Saved: ./run/K3MNIST1/figs/02-many-digits

Step 4 - Create model

About informations about :

• Optimizer
• Activation
• Loss
• Metrics

hidden1     = 100
hidden2     = 100

model = keras.Sequential([
    keras.layers.Input((28,28)),
    keras.layers.Flatten(),
    keras.layers.Dense( hidden1, activation='relu'),
    keras.layers.Dense( hidden2, activation='relu'),
    keras.layers.Dense( 10,      activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

Step 5 - Train the model

batch_size  = 512
epochs      =  16

history = model.fit(  x_train, y_train,
                      batch_size      = batch_size,
                      epochs          = epochs,
                      verbose         = fit_verbosity,
                      validation_data = (x_test, y_test))

Epoch 1/16

118/118 - 1s - 12ms/step - accuracy: 0.8391 - loss: 0.5883 - val_accuracy: 0.9265 - val_loss: 0.2548

Epoch 2/16

118/118 - 1s - 9ms/step - accuracy: 0.9371 - loss: 0.2194 - val_accuracy: 0.9473 - val_loss: 0.1788

Epoch 3/16

118/118 - 1s - 9ms/step - accuracy: 0.9535 - loss: 0.1619 - val_accuracy: 0.9577 - val_loss: 0.1412

Epoch 4/16

118/118 - 1s - 9ms/step - accuracy: 0.9625 - loss: 0.1301 - val_accuracy: 0.9622 - val_loss: 0.1243

Epoch 5/16

118/118 - 1s - 9ms/step - accuracy: 0.9691 - loss: 0.1077 - val_accuracy: 0.9674 - val_loss: 0.1121

Epoch 6/16

118/118 - 1s - 9ms/step - accuracy: 0.9731 - loss: 0.0930 - val_accuracy: 0.9656 - val_loss: 0.1148

Epoch 7/16

118/118 - 1s - 9ms/step - accuracy: 0.9772 - loss: 0.0803 - val_accuracy: 0.9714 - val_loss: 0.0958

Epoch 8/16

118/118 - 1s - 9ms/step - accuracy: 0.9797 - loss: 0.0696 - val_accuracy: 0.9700 - val_loss: 0.0942

Epoch 9/16

118/118 - 1s - 9ms/step - accuracy: 0.9824 - loss: 0.0619 - val_accuracy: 0.9744 - val_loss: 0.0836

Epoch 10/16

118/118 - 1s - 9ms/step - accuracy: 0.9844 - loss: 0.0544 - val_accuracy: 0.9756 - val_loss: 0.0808

Epoch 11/16

118/118 - 1s - 9ms/step - accuracy: 0.9868 - loss: 0.0466 - val_accuracy: 0.9768 - val_loss: 0.0781

Epoch 12/16

118/118 - 1s - 9ms/step - accuracy: 0.9876 - loss: 0.0432 - val_accuracy: 0.9745 - val_loss: 0.0822

Epoch 13/16

118/118 - 1s - 9ms/step - accuracy: 0.9893 - loss: 0.0385 - val_accuracy: 0.9739 - val_loss: 0.0856

Epoch 14/16

118/118 - 1s - 9ms/step - accuracy: 0.9905 - loss: 0.0344 - val_accuracy: 0.9759 - val_loss: 0.0787

Epoch 15/16

118/118 - 1s - 9ms/step - accuracy: 0.9912 - loss: 0.0315 - val_accuracy: 0.9775 - val_loss: 0.0762

Epoch 16/16

118/118 - 1s - 9ms/step - accuracy: 0.9932 - loss: 0.0271 - val_accuracy: 0.9779 - val_loss: 0.0761

Step 6 - Evaluate

6.1 - Final loss and accuracy

score = model.evaluate(x_test, y_test, verbose=0)

print('Test loss     :', score[0])
print('Test accuracy :', score[1])

Test loss     : 0.07613752037286758
Test accuracy : 0.9779000282287598

6.2 - Plot history

fidle.scrawler.history(history, figsize=(6,4), save_as='03-history')

Saved: ./run/K3MNIST1/figs/03-history_0

Saved: ./run/K3MNIST1/figs/03-history_1

6.3 - Plot results

#y_pred   = model.predict_classes(x_test)           Deprecated after 01/01/2021 !!

y_sigmoid = model.predict(x_test, verbose=fit_verbosity)
y_pred    = np.argmax(y_sigmoid, axis=-1)

fidle.scrawler.images(x_test, y_test, range(0,200), columns=12, x_size=1, y_size=1, y_pred=y_pred, save_as='04-predictions')

313/313 - 1s - 2ms/step

Saved: ./run/K3MNIST1/figs/04-predictions

6.4 - Plot some errors

errors=[ i for i in range(len(x_test)) if y_pred[i]!=y_test[i] ]
errors=errors[:min(24,len(errors))]
fidle.scrawler.images(x_test, y_test, errors[:15], columns=6, x_size=2, y_size=2, y_pred=y_pred, save_as='05-some-errors')

Saved: ./run/K3MNIST1/figs/05-some-errors

fidle.scrawler.confusion_matrix(y_test,y_pred,range(10),normalize=True, save_as='06-confusion-matrix')

Saved: ./run/K3MNIST1/figs/06-confusion-matrix

fidle.end()

End time : 22/12/24 21:22:16
Duration : 00:00:47 197ms
This notebook ends here :-)
https://fidle.cnrs.fr

A few things you can do for fun:

• Changing the network architecture (layers, number of neurons, etc.)
• Display a summary of the network
• Retrieve and display the softmax output of the network, to evaluate its "doubts".

---