LSTM model from keras to Pytorch, RayTune hp optimization

Hello there!

I am currently learning how to use LSTMs for a timeseries forecasting algorithm, I have to mention that I it’s my first time using LSTMs. I found and implemented a keras LSTM model and did hp optimization with keras.tuner. However I want to migrate the model to PyTorch framework and the algorithm for hp optimization to RayTune but I really struggle doing so.

Here is my keras LSTM model:

def build_model(hp):
    model = Sequential()
    model.add(LSTM(hp.Int('input_unit',min_value=32,max_value=512,step=32),return_sequences=True, input_shape=(train_X.shape[1],train_X.shape[2])))
    for i in range(hp.Int('n_layers', 2, 6)): 
        model.add(LSTM(hp.Int(f'lstm_{i}_units',min_value=32,max_value=512,step=32),return_sequences=True))
    model.add(LSTM(hp.Int('layer_2_neurons',min_value=32,max_value=512,step=32)))
    model.add(Dropout(hp.Float('Dropout_rate',min_value=0,max_value=0.5,step=0.1)))
    model.add(Dense(train_y.shape[1], activation=hp.Choice('dense_activation',values=['relu', 'sigmoid'],default='relu')))
    model.compile(loss='mean_squared_error', optimizer='adam',metrics = ['mse'])
    return model

and here is the hp tuning I performed with keras.tuner:

tuner= BayesianOptimization (
        build_model,
        objective='mse',
        max_trials=50,
        executions_per_trial=3,
        directory='randomSearchWeekly-lag-features-3-8',
        seed=123
        )
tuner.search(
        x=train_X,
        y=train_y,
        epochs=50,
        batch_size=128,
        validation_data=(test_X, test_y),
)

Next, the X and y train/test shapes:

shape for train_X: (222, 3, 8)
shape for train_y: (222, 1)
shape for test_X: (4, 3, 8)
shape for test_y: (4, 1)

My dataframe contains 229 rows, based on weekly data, with 8 features (including lagged features) for each timestamp. I basically want to predict only one feature for the next week for example, based on the value of previous features with a loopback period of n weeks.

Hello. I am having the same issue. It seems to me like PyTorch is not very intuitive with LSTMs?

Does anybody have any suggestions?

1 Like