[K3LADYB1] - Prediction of a 2D trajectory via RNNĀ¶
Artificial dataset generation and prediction attempt via a recurrent network, using Keras 3 and PyTorchObjectives :Ā¶
- Understanding the use of a recurrent neural network
What we're going to do :Ā¶
- Generate an artificial dataset
- dataset preparation
- Doing our testing
- Making predictions
Step 1 - Import and initĀ¶
1.1 - PythonĀ¶
InĀ [1]:
import os
os.environ['KERAS_BACKEND'] = 'torch'
import keras
import numpy as np
from math import cos, sin
import random
import fidle
# Init Fidle environment
run_id, run_dir, datasets_dir = fidle.init('K3LADYB1')
FIDLE - Environment initialization
Version : 2.3.2 Run id : K3LADYB1 Run dir : ./run/K3LADYB1 Datasets dir : /lustre/fswork/projects/rech/mlh/uja62cb/fidle-project/datasets-fidle Start time : 22/12/24 21:23:33 Hostname : r3i7n1 (Linux) Tensorflow log level : Info + Warning + Error (=0) Update keras cache : False Update torch cache : False Save figs : ./run/K3LADYB1/figs (True) keras : 3.7.0 numpy : 2.1.2 sklearn : 1.5.2 yaml : 6.0.2 matplotlib : 3.9.2 pandas : 2.2.3 torch : 2.5.0
1.2 - ParametersĀ¶
InĀ [2]:
# ---- About dataset
#
max_t = 1000
delta_t = 0.01
features_len = 2
sequence_len = 20
predict_len = 5
# ---- About training
#
scale = .2 # Percentage of dataset to be used (1=all)
train_prop = .8 # Percentage for train (the rest being for the test)
batch_size = 32
epochs = 5
fit_verbosity = 1 # 0 = silent, 1 = progress bar, 2 = one line per epoch
Override parameters (batch mode) - Just forget this cell
InĀ [3]:
fidle.override('scale', 'train_prop', 'sequence_len', 'predict_len', 'batch_size', 'epochs', 'fit_verbosity')
** Overrided parameters : ** scale : 1 train_prop : 0.8 sequence_len : 20 predict_len : 5 batch_size : 32 epochs : 10 fit_verbosity : 2
InĀ [4]:
def ladybug_init(s=122):
if s>0 : random.seed(s)
ladybug_init.params_x = [ random.gauss(0.,1.) for u in range(8)]
ladybug_init.params_y = [ random.gauss(0.,1.) for u in range(8)]
def ladybug_move(t):
[ax1, ax2, ax3, ax4, kx1, kx2, kx3, kx4] = ladybug_init.params_x
[ay1, ay2, ay3, ay4, ky1, ky2, ky3, ky4] = ladybug_init.params_y
x = ax1*sin(t*(kx1+20)) + ax2*cos(t*(kx2+10)) + ax3*sin(t*(kx3+5)) + ax4*cos(t*(kx4+5))
y = ay1*cos(t*(ky1+20)) + ay2*sin(t*(ky2+10)) + ay3*cos(t*(ky3+5)) + ay4*sin(t*(ky4+5))
return x,y
2.2 - Get some positions, and build a rescaled and normalized datasetĀ¶
InĀ [5]:
# ---- Get positions
#
ladybug_init(s=16)
x,y = 0,0
positions=[]
for t in np.arange(0., max_t, delta_t):
x,y = ladybug_move(t)
positions.append([x,y])
# ---- Build rescaled dataset
#
n = int( len(positions)*scale )
dataset = np.array(positions[:n])
k = int(len(dataset)*train_prop)
x_train = dataset[:k]
x_test = dataset[k:]
# ---- Normalize
#
mean = x_train.mean()
std = x_train.std()
x_train = (x_train - mean) / std
x_test = (x_test - mean) / std
print("Dataset generated.")
print("Train shape is : ", x_train.shape)
print("Test shape is : ", x_test.shape)
Dataset generated. Train shape is : (80000, 2) Test shape is : (20000, 2)
2.3 - Have a lookĀ¶
An extract from the data we have: the virtual trajectory of our ladybug
And what we want to predict (in red), from a segment (in blue)
InĀ [6]:
fidle.scrawler.serie_2d(x_train[:1000], figsize=(12,12), lw=1,ms=4,save_as='01-dataset')
Saved: ./run/K3LADYB1/figs/01-dataset
InĀ [7]:
k1,k2 = sequence_len, predict_len
i = random.randint(0,len(x_test)-k1-k2)
j = i+k1
fidle.scrawler.segment_2d( x_test[i:j+k2], x_test[j:j+k2],ms=6, save_as='02-objectives')
Saved: ./run/K3LADYB1/figs/02-objectives
2.4 - Prepare sequences from datasetsĀ¶
InĀ [8]:
# ---- Create sequences and labels for train and test
#
xs_train, ys_train=[],[]
all_i = np.random.permutation( len(x_train) - sequence_len - 1 )
for i in all_i:
xs_train.append( x_train[ i : i+sequence_len ] )
ys_train.append( x_train[ i+sequence_len+1 ] )
xs_test, ys_test=[],[]
for i in range( len(x_test) - sequence_len - 1):
xs_test.append( x_test[ i : i+sequence_len ] )
ys_test.append( x_test[ i+sequence_len+1 ] )
# ---- Convert to numpy / float16
xs_train = np.array(xs_train, dtype='float16')
ys_train = np.array(ys_train, dtype='float16')
xs_test = np.array(xs_test, dtype='float16')
ys_test = np.array(ys_test, dtype='float16')
InĀ [9]:
fidle.utils.subtitle('About the splitting of our dataset :')
print('Number of sequences : ', len(xs_train))
print('xs_train shape : ',xs_train.shape)
print('ys_train shape : ',ys_train.shape)
fidle.utils.subtitle('What an xs look like :')
fidle.utils.np_print(xs_train[10] )
fidle.utils.subtitle('What an ys look like :')
fidle.utils.np_print(ys_train[10])
About the splitting of our dataset :
Number of sequences : 79979 xs_train shape : (79979, 20, 2) ys_train shape : (79979, 2)
What an xs look like :
[[-0.706 0.818] [-0.482 0.786] [-0.247 0.72 ] [-0.006 0.622] [ 0.237 0.494] [ 0.475 0.343] [ 0.703 0.173] [ 0.916 -0.007] [ 1.107 -0.192] [ 1.274 -0.374] [ 1.411 -0.546] [ 1.516 -0.702] [ 1.586 -0.835] [ 1.62 -0.94 ] [ 1.618 -1.013] [ 1.581 -1.052] [ 1.511 -1.054] [ 1.411 -1.02 ] [ 1.284 -0.951] [ 1.137 -0.85 ]]
What an ys look like :
[ 0.797 -0.568]
Step 3 - Create a modelĀ¶
InĀ [10]:
model = keras.models.Sequential()
model.add( keras.layers.InputLayer(shape=(sequence_len, features_len)) )
model.add( keras.layers.GRU(200, return_sequences=False, activation='relu') )
model.add( keras.layers.Dense(features_len) )
model.summary()
model.compile(optimizer='rmsprop',
loss='mse',
metrics = ['mae'] )
Model: "sequential"
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā³āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā³āāāāāāāāāāāāāāāāāā ā Layer (type) ā Output Shape ā Param # ā ā”āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā© ā gru (GRU) ā (None, 200) ā 122,400 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā¼āāāāāāāāāāāāāāāāāā¤ ā dense (Dense) ā (None, 2) ā 402 ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā“āāāāāāāāāāāāāāāāāāāāāāāāāāāāāā“āāāāāāāāāāāāāāāāāā
Total params: 122,802 (479.70 KB)
Trainable params: 122,802 (479.70 KB)
Non-trainable params: 0 (0.00 B)
InĀ [11]:
os.makedirs(f'{run_dir}/models', mode=0o750, exist_ok=True)
save_dir = f'{run_dir}/models/best_model.keras'
savemodel_callback = keras.callbacks.ModelCheckpoint( filepath=save_dir, monitor='val_mae', mode='max', save_best_only=True)
4.2 - Train itĀ¶
Need 3' on a cpu laptop
InĀ [12]:
chrono=fidle.Chrono()
chrono.start()
history=model.fit(xs_train,ys_train,
epochs = epochs,
verbose = fit_verbosity,
validation_data = (xs_test, ys_test),
callbacks = [savemodel_callback])
chrono.show()
Epoch 1/10
2500/2500 - 96s - 38ms/step - loss: 0.0131 - mae: 0.0660 - val_loss: 0.0021 - val_mae: 0.0359
Epoch 2/10
2500/2500 - 96s - 38ms/step - loss: 0.0013 - mae: 0.0271 - val_loss: 4.4078e-04 - val_mae: 0.0161
Epoch 3/10
2500/2500 - 96s - 38ms/step - loss: 8.1212e-04 - mae: 0.0220 - val_loss: 0.0013 - val_mae: 0.0310
Epoch 4/10
2500/2500 - 96s - 38ms/step - loss: 6.1813e-04 - mae: 0.0192 - val_loss: 5.0057e-04 - val_mae: 0.0170
Epoch 5/10
2500/2500 - 96s - 38ms/step - loss: 5.0819e-04 - mae: 0.0174 - val_loss: 2.9932e-04 - val_mae: 0.0132
Epoch 6/10
2500/2500 - 96s - 38ms/step - loss: 4.3076e-04 - mae: 0.0161 - val_loss: 2.1721e-04 - val_mae: 0.0114
Epoch 7/10
2500/2500 - 96s - 38ms/step - loss: 3.8103e-04 - mae: 0.0151 - val_loss: 4.5574e-04 - val_mae: 0.0162
Epoch 8/10
2500/2500 - 96s - 38ms/step - loss: 3.4416e-04 - mae: 0.0143 - val_loss: 1.8212e-04 - val_mae: 0.0102
Epoch 9/10
2500/2500 - 96s - 38ms/step - loss: 3.1525e-04 - mae: 0.0137 - val_loss: 7.3849e-04 - val_mae: 0.0213
Epoch 10/10
2500/2500 - 96s - 38ms/step - loss: 2.8407e-04 - mae: 0.0130 - val_loss: 2.6201e-04 - val_mae: 0.0124
Duration : 961.88 seconds
InĀ [13]:
fidle.scrawler.history(history,plot={'loss':['loss','val_loss'], 'mae':['mae','val_mae']}, save_as='03-history')
Saved: ./run/K3LADYB1/figs/03-history_0
Saved: ./run/K3LADYB1/figs/03-history_1
Step 5 - PredictĀ¶
5.1 - Load modelĀ¶
InĀ [14]:
loaded_model = keras.models.load_model(f'{run_dir}/models/best_model.keras')
print('Loaded.')
Loaded.
5.2 - Make a 1-step predictionĀ¶
A simple prediction on a single iteration
InĀ [15]:
s=random.randint(0,len(x_test)-sequence_len)
sequence = x_test[s:s+sequence_len]
sequence_true = x_test[s:s+sequence_len+1]
sequence_pred = loaded_model.predict( np.array([sequence]), verbose=fit_verbosity )
print('sequence shape :',sequence.shape)
print('sequence true shape :',sequence_true.shape)
print('sequence pred shape :',sequence_pred.shape)
fidle.scrawler.segment_2d(sequence_true, sequence_pred, save_as='04-one-step-prediction')
fidle.scrawler.multivariate_serie(sequence_true, predictions=sequence_pred, labels=['Axis=0', 'Axis=1'],save_as='05-one-step-prediction-2axis')
1/1 - 0s - 23ms/step
sequence shape : (20, 2) sequence true shape : (21, 2) sequence pred shape : (1, 2)
Saved: ./run/K3LADYB1/figs/04-one-step-prediction
Saved: ./run/K3LADYB1/figs/05-one-step-prediction-2axis
5.3 - Make n-steps predictionĀ¶
A longer term prediction, via a nice iteration function
We will perform
InĀ [16]:
def get_prediction(dataset, model, iterations=4):
# ---- Initial sequence
#
s=random.randint(0,len(dataset)-sequence_len-iterations)
sequence_pred = dataset[s:s+sequence_len].copy()
sequence_true = dataset[s:s+sequence_len+iterations].copy()
# ---- Iterate
#
sequence_pred = list(sequence_pred)
for i in range(iterations):
sequence = sequence_pred[-sequence_len:]
prediction = model.predict( np.array([sequence]), verbose=fit_verbosity )
sequence_pred.append(prediction[0])
# ---- Extract the predictions
#
prediction = np.array(sequence_pred[-iterations:])
return sequence_true,prediction
An n-steps prediction :
InĀ [17]:
sequence_true, sequence_pred = get_prediction(x_test, loaded_model, iterations=5)
fidle.scrawler.segment_2d(sequence_true, sequence_pred, ms=8, save_as='06-n-steps-prediction-norm')
fidle.scrawler.multivariate_serie(sequence_true, predictions=sequence_pred, hide_ticks=True, labels=['Axis=0', 'Axis=1'],save_as='07-n-steps-prediction-norm')
1/1 - 0s - 20ms/step
1/1 - 0s - 18ms/step
1/1 - 0s - 18ms/step
1/1 - 0s - 18ms/step
1/1 - 0s - 20ms/step
Saved: ./run/K3LADYB1/figs/06-n-steps-prediction-norm
Saved: ./run/K3LADYB1/figs/07-n-steps-prediction-norm
InĀ [18]:
fidle.end()
End time : 22/12/24 21:39:39
Duration : 00:16:07 711ms
This notebook ends here :-)
https://fidle.cnrs.fr
