Pytorch Lightening prediction output length does not make sense

1

Hi everyone,
I have trained an NBeats model from PyTorch forecasting using [TimeSeriesDataSet — pytorch-forecasting documentation](https://Pythorch Forecasting)
using TimeSeriesDataSet method.

Here is the configuration of model and data:

training = TimeSeriesDataSet(
   train_data,
    time_idx="time",
    target="target",
    group_ids=["group"],
    time_varying_unknown_reals=["target"],
    max_encoder_length=100,
    max_prediction_length=100
)

train_load= training.to_dataloader(train=True, batch_size=128)

test_load= testing.to_dataloader(train=False, batch_size=128)

My goal is to look back the past data points and forecast the next 100 points in future, and this is why I set the max_prediction = 100.

The test data has 234 samples.

I tried to look at the predicted value but the size of prediction is 35 and not 100 and this is super strange since I want to forecast the next 100 (the prediction_horizon).
The prediction is:

Pred = mymodel.predict(dataloaders=test_load)

len(Pred)

35

Also if you look at the

print(test_load.dataset)

TimeSeriesDataSet[length=35](...)

You see the output also shows the length is 35 and not 100.
Can someone please explain why I get 35 as the length and not 100:

2

The length of a tensor is generally just the extent of the first dimension:

>>> a = torch.empty(2, 3, 1)
>>> len(a)
2
>>> a = torch.empty(3, 2,  1)
>>> len(a)
3
>>> a = torch.empty(1, 2)
>>> len(a)
1

so len(Pred) is not a useful indicator for the number of forecasted steps in your case unless it was the first dimension. Instead it looks like the batch size is the first dimension, as it matches the “length” of your dataset. You can check which dimension corresponds to the “steps” by printing out the shape of the prediction rather than just the length. I would also double check the number of samples in your dataset, as it having length 35 suggests that there are only 35 samples.