No description has been provided for this image

[K3MNIST1] - Simple classification with DNN¶

An example of classification using a dense neural network for the famous MNIST dataset

Objectives :¶

  • Recognizing handwritten numbers
  • Understanding the principle of a classifier DNN network
  • Implementation with Keras

The MNIST dataset (Modified National Institute of Standards and Technology) is a must for Deep Learning.
It consists of 60,000 small images of handwritten numbers for learning and 10,000 for testing.

What we're going to do :¶

  • Retrieve data
  • Preparing the data
  • Create a model
  • Train the model
  • Evaluate the result

Step 1 - Init python stuff¶

In [1]:
import os
os.environ['KERAS_BACKEND'] = 'torch'

import keras

import numpy as np
import matplotlib.pyplot as plt
import sys,os
from importlib import reload

# Init Fidle environment
import fidle

run_id, run_dir, datasets_dir = fidle.init('K3MNIST1')


FIDLE - Environment initialization

Version              : 2.3.0
Run id               : K3MNIST1
Run dir              : ./run/K3MNIST1
Datasets dir         : /gpfswork/rech/mlh/uja62cb/fidle-project/datasets-fidle
Start time           : 03/03/24 21:04:03
Hostname             : r3i6n3 (Linux)
Tensorflow log level : Warning + Error  (=1)
Update keras cache   : False
Update torch cache   : False
Save figs            : ./run/K3MNIST1/figs (True)
keras                : 3.0.4
numpy                : 1.24.4
sklearn              : 1.3.2
yaml                 : 6.0.1
matplotlib           : 3.8.2
pandas               : 2.1.3
torch                : 2.1.1

Verbosity during training : 0 = silent, 1 = progress bar, 2 = one line per epoch

In [2]:
fit_verbosity = 1

Override parameters (batch mode) - Just forget this cell

In [3]:
fidle.override('fit_verbosity')

** Overrided parameters : **
fit_verbosity        : 2

Step 2 - Retrieve data¶

MNIST is one of the most famous historic dataset.
Include in Keras datasets

In [4]:
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

print("x_train : ",x_train.shape)
print("y_train : ",y_train.shape)
print("x_test  : ",x_test.shape)
print("y_test  : ",y_test.shape)

x_train :  (60000, 28, 28)
y_train :  (60000,)
x_test  :  (10000, 28, 28)
y_test  :  (10000,)

Step 3 - Preparing the data¶

In [5]:
print('Before normalization : Min={}, max={}'.format(x_train.min(),x_train.max()))

xmax=x_train.max()
x_train = x_train / xmax
x_test  = x_test  / xmax

print('After normalization  : Min={}, max={}'.format(x_train.min(),x_train.max()))

Before normalization : Min=0, max=255
After normalization  : Min=0.0, max=1.0

Have a look¶

In [6]:
fidle.scrawler.images(x_train, y_train, [27],  x_size=5,y_size=5, colorbar=True, save_as='01-one-digit')
fidle.scrawler.images(x_train, y_train, range(5,41), columns=12, save_as='02-many-digits')
Saved: ./run/K3MNIST1/figs/01-one-digit
No description has been provided for this image
Saved: ./run/K3MNIST1/figs/02-many-digits
No description has been provided for this image

Step 4 - Create model¶

About informations about :

  • Optimizer
  • Activation
  • Loss
  • Metrics
In [7]:
hidden1     = 100
hidden2     = 100

model = keras.Sequential([
    keras.layers.Input((28,28)),
    keras.layers.Flatten(),
    keras.layers.Dense( hidden1, activation='relu'),
    keras.layers.Dense( hidden2, activation='relu'),
    keras.layers.Dense( 10,      activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

Step 5 - Train the model¶

In [8]:
batch_size  = 512
epochs      =  16

history = model.fit(  x_train, y_train,
                      batch_size      = batch_size,
                      epochs          = epochs,
                      verbose         = fit_verbosity,
                      validation_data = (x_test, y_test))

Epoch 1/16
118/118 - 9s - 74ms/step - accuracy: 0.8436 - loss: 0.5869 - val_accuracy: 0.9285 - val_loss: 0.2528
Epoch 2/16
118/118 - 8s - 70ms/step - accuracy: 0.9388 - loss: 0.2162 - val_accuracy: 0.9469 - val_loss: 0.1861
Epoch 3/16
118/118 - 8s - 70ms/step - accuracy: 0.9528 - loss: 0.1622 - val_accuracy: 0.9556 - val_loss: 0.1492
Epoch 4/16
118/118 - 8s - 71ms/step - accuracy: 0.9624 - loss: 0.1305 - val_accuracy: 0.9607 - val_loss: 0.1315
Epoch 5/16
118/118 - 8s - 70ms/step - accuracy: 0.9678 - loss: 0.1107 - val_accuracy: 0.9655 - val_loss: 0.1154
Epoch 6/16
118/118 - 8s - 70ms/step - accuracy: 0.9732 - loss: 0.0910 - val_accuracy: 0.9667 - val_loss: 0.1089
Epoch 7/16
118/118 - 8s - 71ms/step - accuracy: 0.9768 - loss: 0.0796 - val_accuracy: 0.9681 - val_loss: 0.1057
Epoch 8/16
118/118 - 8s - 70ms/step - accuracy: 0.9796 - loss: 0.0699 - val_accuracy: 0.9701 - val_loss: 0.0964
Epoch 9/16
118/118 - 8s - 70ms/step - accuracy: 0.9822 - loss: 0.0604 - val_accuracy: 0.9728 - val_loss: 0.0872
Epoch 10/16
118/118 - 8s - 71ms/step - accuracy: 0.9847 - loss: 0.0527 - val_accuracy: 0.9729 - val_loss: 0.0877
Epoch 11/16
118/118 - 8s - 70ms/step - accuracy: 0.9865 - loss: 0.0470 - val_accuracy: 0.9733 - val_loss: 0.0929
Epoch 12/16
118/118 - 8s - 70ms/step - accuracy: 0.9876 - loss: 0.0423 - val_accuracy: 0.9726 - val_loss: 0.0867
Epoch 13/16
118/118 - 8s - 70ms/step - accuracy: 0.9887 - loss: 0.0385 - val_accuracy: 0.9752 - val_loss: 0.0843
Epoch 14/16
118/118 - 8s - 71ms/step - accuracy: 0.9905 - loss: 0.0343 - val_accuracy: 0.9755 - val_loss: 0.0853
Epoch 15/16
118/118 - 8s - 70ms/step - accuracy: 0.9915 - loss: 0.0306 - val_accuracy: 0.9745 - val_loss: 0.0851
Epoch 16/16
118/118 - 8s - 71ms/step - accuracy: 0.9929 - loss: 0.0258 - val_accuracy: 0.9741 - val_loss: 0.0847

Step 6 - Evaluate¶

6.1 - Final loss and accuracy¶

In [9]:
score = model.evaluate(x_test, y_test, verbose=0)

print('Test loss     :', score[0])
print('Test accuracy :', score[1])

Test loss     : 0.08292659372091293
Test accuracy : 0.9740999937057495

6.2 - Plot history¶

In [10]:
fidle.scrawler.history(history, figsize=(6,4), save_as='03-history')
Saved: ./run/K3MNIST1/figs/03-history_0
No description has been provided for this image
Saved: ./run/K3MNIST1/figs/03-history_1
No description has been provided for this image

6.3 - Plot results¶

In [11]:
#y_pred   = model.predict_classes(x_test)           Deprecated after 01/01/2021 !!

y_sigmoid = model.predict(x_test, verbose=fit_verbosity)
y_pred    = np.argmax(y_sigmoid, axis=-1)

fidle.scrawler.images(x_test, y_test, range(0,200), columns=12, x_size=1, y_size=1, y_pred=y_pred, save_as='04-predictions')

313/313 - 1s - 3ms/step
Saved: ./run/K3MNIST1/figs/04-predictions
No description has been provided for this image

6.4 - Plot some errors¶

In [12]:
errors=[ i for i in range(len(x_test)) if y_pred[i]!=y_test[i] ]
errors=errors[:min(24,len(errors))]
fidle.scrawler.images(x_test, y_test, errors[:15], columns=6, x_size=2, y_size=2, y_pred=y_pred, save_as='05-some-errors')
Saved: ./run/K3MNIST1/figs/05-some-errors
No description has been provided for this image
In [13]:
fidle.scrawler.confusion_matrix(y_test,y_pred,range(10),normalize=True, save_as='06-confusion-matrix')
Saved: ./run/K3MNIST1/figs/06-confusion-matrix
No description has been provided for this image
In [14]:
fidle.end()

End time : 03/03/24 21:06:49
Duration : 00:02:46 223ms
This notebook ends here :-)
https://fidle.cnrs.fr

A few things you can do for fun:
  • Changing the network architecture (layers, number of neurons, etc.)
  • Display a summary of the network
  • Retrieve and display the softmax output of the network, to evaluate its "doubts".

No description has been provided for this image