I am planning to use CNN for a task of univariate time series forecasting. I am using the following settings for analysis, but I am having trouble with the error shown in the title.
class TimeseriesDataset(torch.utils.data.Dataset):
def __init__(self, X, y, seq_len=1):
self.X = X
self.y = y
self.seq_len = seq_len
def __len__(self):
return self.X.__len__() - (self.seq_len-1)
def __getitem__(self, index):
return (self.X[index:index+self.seq_len], self.y[index+self.seq_len])
class CNNForecast(nn.Module):
def __init__(self, w, p_w):
super(CNNForecast, self).__init__()
self.first_conv1d = nn.Conv1d(in_channels=w, out_channels=32, kernel_size=2, stride=1, padding=0)
self.maxpool1d = nn.MaxPool1d(kernel_size=2, stride=2)
self.second_conv1d = nn.Conv1d(in_channels=50, out_channels=32, kernel_size=2, stride=1, padding=0)
self.flatten = nn.Flatten()
self.linear = nn.Linear(32*2, 40)
self.dropout = nn.Dropout(p=0.5)
self.out = nn.Linear(40, 1)
def forward(self, x):
x = F.relu(self.first_conv1d(x))
x = self.maxpool1d(x)
x = F.relu(self.second_conv1d(x))
x = self.maxpool1d(x)
x = self.flatten(x)
x = self.dropout(x)
x = self.out(x)
return x
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model = CNNForecast(w=100, p_w=1).to(device)
optimizer = torch.optim.Adam(model.parameters(), lr=2e-5)
criterion = nn.L1Loss()
train_dataset = TimeseriesDataset(df['value'].values, df['value'].values, seq_len=100)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=False)
train_losses = []
model.train()
for epoch in range(30):
epoch_loss = 0
for batch, (inputs, labels) in enumerate(train_loader):
inputs = inputs.to(device)
labels = labels.to(device)
print(inputs.size())
print('-----')
print(labels.size())
optimizer.zero_grad()
preds = model(inputs.float())
loss = criterion(preds, labels)
loss.backward()
optimizer.step()
Something seems strange about the data loading vs. the weights here. If the weights to the first layer have 32 input channels in_channels=w, then the input should also have 32 channels. However, it looks like the 32 in the input is actually the “batch” dimension, with the channel dimension missing (as 100 is the sequence length dimension).
You can try reducing the number of input channels for the convolution to just 1, and reshaping the input to [32, 1, 100] in this case, or change the construction of the TimeseriesDataset so the input has two channels to match the first convolution layer.
You can read the details of the expected input and output shapes for a 1d convolution here. Note that the input needs to be three dimensional, and match the number of input channels to the convolution.
Right, for that to work, you must also change your w parameter to change the input channels from 100 to 1. With an input layout of 32, 1, 100, the sequence length is 100, but the number of channels is 1.
The 2 comes from the kernel_size (e.g., the 1d convolution that is two time steps wide).
The issue is that each layer’s input must be compatible with the previous layer’s output. For example, your first convolution has 32 output channels, but the second convolution expects just a single input channel. These numbers need to match.
It is a similar issue to before; the input shape to the first linear layer must be [?, 32] according to the current code. You can compute (or simply print the shape before the linear layer) to see what the shape of the linear layer should be changed to. However, I don’t see a call to the linear layer in your forward function so I’m not sure if the code posted corresponds to the error.
The docs for linear layer show a good example of how the input shape should be matched to the linear layer.