Hi everyone, i am pretty new in the Pytorch world, and in 1D convolution. I am working with some time series data, and i am trying to make a convolutive neural network that predicts the next value, given a window size of for example 10. So i want my model to train so that given 10 time steps in input, it predicts the next value at time step t+1. To begin i started with a simple toy dataset, generating 400 datapoints of a sine curve and windowing the data so that the format should look like this.
input window:
tensor([[-0.4532],
[ 0.9988],
[ 2.3085],
[ 3.2892],
[ 3.8014],
[ 3.7718],
[ 3.2049],
[ 2.1813],
[ 0.8469],
[-0.6081]])
output:
tensor([-1.9765])
Considering that this is an univariate time series, window lenght of 10 and 390 (400-10) data to train with, in order to use the convolution in the appropriate way, what should i put in the parameters in the Conv1D function? The model i created gives really good results, but my issue is that i think that maybe it is just because the dataset is very simple and he is not actually using the convolution but just learning from each point individually.
My model is this:
class CNN1d(nn.Module):
def __init__(self, input_size, output_size):
super(CNN1d, self).__init__()
self.conv1 = nn.Conv1d(input_size, 64, kernel_size=3, stride=1, padding=2)
self.apply(init_params) #function to initiate model parameters
self.pool1 = nn.MaxPool1d(2)
self.conv2 = nn.Conv1d(64,64,kernel_size=3, stride = 1, padding=1)
self.fc1 = nn.Linear(64, output_size)
self.fc2 = nn.Linear(1, 1)
def forward(self, x):
in_size1 = x.size(0) # one batch
x = F.relu(self.conv1(x))
x = self.pool1(x)
x = F.relu(self.conv2(x))
x = x.view(in_size1, -1) # flatten the tensor
x = self.fc1(x)
output = self.fc2(x)
return output
And this is how i train it:
loss_fn = nn.MSELoss()
EPOCH = 300
train_losses = [] # Collect training losses
for epoch in range(EPOCH):
train_pred = cnn1d(X)
train_loss = loss_fn(train_pred, y)
train_losses.append(train_loss)
optimizer.zero_grad()
train_loss.backward()
optimizer.step()
The shape of the X i am passing is:
torch.Size([390, 10, 1])
where 390 is the dimension of the training set, 10 is the time steps, which i interpreted as the window size and 1 is the number of features, as i am using a univariate time series.
The output_size in the end is 1 as i am trying to predict a single value, but what should the input size be? should it be 1 as there is only one single variable or 10 which is the size of the window on which i am training? And how is the convoution moving across my data?
For conext, my model does very well in predicting the values in the time steps of the test sets, which are the continuation of the training ones, and i use as input_size the size of the window, but is this correct?
I am sorry if my questions might seem trivial, but i am having some difficulties understanding why and how this works