[K3AE3] - Playing with our denoiser model

Episode 2 : Using the previously trained autoencoder to denoise data

Objectives :

• Retrieve and use our denoiser model

What we're going to do :

• Reload our dataset and saved best model
• Encode/decode some test images (neved used, never seen by the model)

Data Terminology :

• clean_train, clean_test for noiseless images
• noisy_train, noisy_test for noisy images
• denoised_test for denoised images at the output of the model

Step 1 - Init python stuff

1.1 - Init

import os
os.environ['KERAS_BACKEND'] = 'torch'

import keras

import numpy as np
import matplotlib.pyplot as plt
import random

from modules.MNIST import MNIST

import fidle

# Init Fidle environment
run_id, run_dir, datasets_dir = fidle.init('K3AE3')

FIDLE - Environment initialization

Version              : 2.3.0
Run id               : K3AE3
Run dir              : ./run/K3AE3
Datasets dir         : /gpfswork/rech/mlh/uja62cb/fidle-project/datasets-fidle
Start time           : 03/03/24 21:13:10
Hostname             : r6i1n1 (Linux)
Tensorflow log level : Warning + Error  (=1)
Update keras cache   : False
Update torch cache   : False
Save figs            : ./run/K3AE3/figs (True)
keras                : 3.0.4
numpy                : 1.24.4
sklearn              : 1.3.2
yaml                 : 6.0.1
matplotlib           : 3.8.2
pandas               : 2.1.3
torch                : 2.1.1

1.2 - Parameters

These parameters must be identical to those used during the training in order to have the same dataset.
prepared_dataset : Filename of the prepared dataset (Need 400 Mo, but can be in ./data)
dataset_seed : Random seed for shuffling dataset
scale : % of the dataset to use (1. for 100%)
train_prop : Percentage for train (the rest being for the test)

prepared_dataset = './data/mnist-noisy.h5'
saved_models     = './run/K3AE2/models'
dataset_seed     = 123
scale            = 1
train_prop       = .8

Override parameters (batch mode) - Just forget this cell

fidle.override('prepared_dataset', 'dataset_seed', 'scale', 'train_prop')

Step 2 - Retrieve dataset

With our MNIST class, in one call, we can reload, rescale, shuffle and split our previously saved dataset :-)
Important : Make sure that the digest is identical to the one used during the training !
See : AE2 / Step 2 - Retrieve dataset

clean_train,clean_test, noisy_train,noisy_test, _,_ = MNIST.reload_prepared_dataset(scale      = scale, 
                                                                                    train_prop = train_prop,
                                                                                    seed       = dataset_seed,
                                                                                    shuffle    = True,
                                                                                    filename=prepared_dataset )

Loaded.
rescaled (1).
Seeded (123)
Shuffled.
splited (0.8).
clean_train shape is :  (56000, 28, 28, 1)
clean_test  shape is :  (14000, 28, 28, 1)
noisy_train shape is :  (56000, 28, 28, 1)
noisy_test  shape is :  (14000, 28, 28, 1)
class_train shape is :  (56000,)
class_test  shape is :  (14000,)
Blake2b digest is    :  de2af55afacf9fb3ee93

Step 3 - Evaluation

Note : We will use the following data:
clean_train, clean_test for noiseless images
noisy_train, noisy_test for noisy images
denoised_test for denoised images at the output of the model

3.1 - Reload our best model

# model = keras.models.load_model(f'{saved_models}/model.keras')

encoder = keras.models.load_model(f'{saved_models}/encoder.keras')
decoder = keras.models.load_model(f'{saved_models}/decoder.keras')

inputs    = keras.Input(shape=(28, 28, 1))

latents   = encoder(inputs)
outputs   = decoder(latents)

model = keras.Model(inputs,outputs, name="ae")

3.2 - Let's make a prediction

from tabnanny import verbose


denoised_test = model.predict(noisy_test,verbose=0)

print('Denoised images   (denoised_test) shape : ',denoised_test.shape)

/gpfswork/rech/mlh/uja62cb/local/fidle-k3/lib/python3.11/site-packages/keras/src/backend/common/backend_utils.py:88: UserWarning: You might experience inconsistencies accross backends when calling conv transpose with kernel_size=3, stride=2, dilation_rate=1, padding=same, output_padding=1.
  warnings.warn(

Denoised images   (denoised_test) shape :  (14000, 28, 28, 1)

3.3 - Denoised images

i=random.randint(0,len(denoised_test)-8)
j=i+8

fidle.utils.subtitle('Noisy test images (input):')
fidle.scrawler.images(noisy_test[i:j], None, indices='all', columns=8, x_size=2,y_size=2, interpolation=None, save_as='05-test-noisy')

fidle.utils.subtitle('Denoised images (output):')
fidle.scrawler.images(denoised_test[i:j], None, indices='all', columns=8, x_size=2,y_size=2, interpolation=None, save_as='06-test-predict')

fidle.utils.subtitle('Real test images :')
fidle.scrawler.images(clean_test[i:j], None, indices='all', columns=8, x_size=2,y_size=2, interpolation=None, save_as='07-test-real')

Noisy test images (input):

Saved: ./run/K3AE3/figs/05-test-noisy

Denoised images (output):

Saved: ./run/K3AE3/figs/06-test-predict

Real test images :

Saved: ./run/K3AE3/figs/07-test-real

Step 4 - Looking at the latent space

4.1 - Getting clean data and class

clean_data,_, _,_, class_data,_ = MNIST.reload_prepared_dataset(scale      = 1, 
                                                                train_prop = 1,
                                                                seed       = dataset_seed,
                                                                shuffle    = False,
                                                                filename   = prepared_dataset )

Loaded.
rescaled (1).
Seeded (123)
splited (1).
clean_train shape is :  (70000, 28, 28, 1)
clean_test  shape is :  (0, 28, 28, 1)
noisy_train shape is :  (70000, 28, 28, 1)
noisy_test  shape is :  (0, 28, 28, 1)
class_train shape is :  (70000,)
class_test  shape is :  (0,)
Blake2b digest is    :  85a22e6bb958c12c88f6

4.2 - Retrieve encoder

encoder=model.get_layer('encoder')

4.3 Showing latent space

Here is the digit distribution in the latent space

n_show = 20000

# ---- Select images

x_show, y_show = fidle.utils.pick_dataset(clean_data, class_data, n=n_show)

# ---- Get latent points

z = encoder.predict(x_show)

# ---- Show them

fig = plt.figure(figsize=(14, 10))
plt.scatter(z[:, 0] , z[:, 1], c=y_show, cmap= 'tab10', alpha=0.5, s=30)
plt.colorbar()
fidle.scrawler.save_fig('08-Latent-space')
plt.show()

625/625 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step

Saved: ./run/K3AE3/figs/08-Latent-space

fidle.end()

End time : 03/03/24 21:13:22
Duration : 00:00:12 197ms
This notebook ends here :-)
https://fidle.cnrs.fr

---