Hello there!
I am currently learning how to use LSTMs for a timeseries forecasting algorithm, I have to mention that I it’s my first time using LSTMs. I found and implemented a keras LSTM model and did hp optimization with keras.tuner. However I want to migrate the model to PyTorch framework and the algorithm for hp optimization to RayTune but I really struggle doing so.
Here is my keras LSTM model:
def build_model(hp):
model = Sequential()
model.add(LSTM(hp.Int('input_unit',min_value=32,max_value=512,step=32),return_sequences=True, input_shape=(train_X.shape[1],train_X.shape[2])))
for i in range(hp.Int('n_layers', 2, 6)):
model.add(LSTM(hp.Int(f'lstm_{i}_units',min_value=32,max_value=512,step=32),return_sequences=True))
model.add(LSTM(hp.Int('layer_2_neurons',min_value=32,max_value=512,step=32)))
model.add(Dropout(hp.Float('Dropout_rate',min_value=0,max_value=0.5,step=0.1)))
model.add(Dense(train_y.shape[1], activation=hp.Choice('dense_activation',values=['relu', 'sigmoid'],default='relu')))
model.compile(loss='mean_squared_error', optimizer='adam',metrics = ['mse'])
return model
and here is the hp tuning I performed with keras.tuner:
tuner= BayesianOptimization (
build_model,
objective='mse',
max_trials=50,
executions_per_trial=3,
directory='randomSearchWeekly-lag-features-3-8',
seed=123
)
tuner.search(
x=train_X,
y=train_y,
epochs=50,
batch_size=128,
validation_data=(test_X, test_y),
)
Next, the X and y train/test shapes:
shape for train_X: (222, 3, 8)
shape for train_y: (222, 1)
shape for test_X: (4, 3, 8)
shape for test_y: (4, 1)
My dataframe contains 229 rows, based on weekly data, with 8 features (including lagged features) for each timestamp. I basically want to predict only one feature for the next week for example, based on the value of previous features with a loopback period of n weeks.