Correct way of using Conv1d on time series data

Hi everyone, i am pretty new in the Pytorch world, and in 1D convolution. I am working with some time series data, and i am trying to make a convolutive neural network that predicts the next value, given a window size of for example 10. So i want my model to train so that given 10 time steps in input, it predicts the next value at time step t+1. To begin i started with a simple toy dataset, generating 400 datapoints of a sine curve and windowing the data so that the format should look like this.

input window:

tensor([[-0.4532],
[ 0.9988],
[ 2.3085],
[ 3.2892],
[ 3.8014],
[ 3.7718],
[ 3.2049],
[ 2.1813],
[ 0.8469],
[-0.6081]])

output:
tensor([-1.9765])

Considering that this is an univariate time series, window lenght of 10 and 390 (400-10) data to train with, in order to use the convolution in the appropriate way, what should i put in the parameters in the Conv1D function? The model i created gives really good results, but my issue is that i think that maybe it is just because the dataset is very simple and he is not actually using the convolution but just learning from each point individually.

My model is this:

class CNN1d(nn.Module):
    def __init__(self, input_size, output_size):
        super(CNN1d, self).__init__()

        self.conv1 = nn.Conv1d(input_size, 64, kernel_size=3, stride=1, padding=2)
        self.apply(init_params)  #function to initiate model parameters
        self.pool1 = nn.MaxPool1d(2)
        self.conv2 = nn.Conv1d(64,64,kernel_size=3, stride = 1, padding=1)

        self.fc1 = nn.Linear(64, output_size)
        self.fc2 = nn.Linear(1, 1)

    def forward(self, x):
        in_size1 = x.size(0)  # one batch
        x = F.relu(self.conv1(x))
        x = self.pool1(x)
        x = F.relu(self.conv2(x))
        x = x.view(in_size1, -1)  # flatten the tensor
        x = self.fc1(x)
        output = self.fc2(x)

        return output

And this is how i train it:

loss_fn = nn.MSELoss()

EPOCH = 300
train_losses = [] # Collect training losses

for epoch in range(EPOCH):

    train_pred = cnn1d(X)
    train_loss = loss_fn(train_pred, y)
    train_losses.append(train_loss)

    optimizer.zero_grad()
    train_loss.backward()
    optimizer.step()

The shape of the X i am passing is:
torch.Size([390, 10, 1])
where 390 is the dimension of the training set, 10 is the time steps, which i interpreted as the window size and 1 is the number of features, as i am using a univariate time series.

The output_size in the end is 1 as i am trying to predict a single value, but what should the input size be? should it be 1 as there is only one single variable or 10 which is the size of the window on which i am training? And how is the convoution moving across my data?
For conext, my model does very well in predicting the values in the time steps of the test sets, which are the continuation of the training ones, and i use as input_size the size of the window, but is this correct?
I am sorry if my questions might seem trivial, but i am having some difficulties understanding why and how this works

It is incorrect to do unrolling manually, kernel_size is like window size.

Conv1d(1,ch,kernel_size=10) on tensor [batch=1, channels=1, time=400]

does the same as

nn.Linear(10,ch) on an unrolled tensor [time=390,window=10]

You can sometimes interpret such inputs as [batch=390, channels=1, time=10], but it is not the usual case, esp. with overlapping.

Thank you for the reply,
Unfortunately I did not understand clearly what you said. So as i was afraid i am not convolving in the right way over the dataset right? But in time series forecasting, i saw in every tutorial that the data should be split in windows so that you use the first n samples to predict the sample at time n+1, is the data format wrong too? Sorry for bothering, but could you tell me how should i organize my data and set the convolution parameters in order to perform the convolution correctly?

You’d create data windows for sampling purposes, but Conv & RNN inputs have no “window” dimension - it is just “time”.

With your toy problem, different batching/sampling schemes would work, but general approach would be:

  1. sample batch_size sequences of length t
  2. use input shape [batch, channels_in=1, t]
  3. in general, you can predict multiple values in parallel (one step into the future), and this is the natural approach with convolutions. but it is possible to use many-to-one mode too (keywords: dilated causal convolutions).
1 Like

Thank you, i think i understood.