RuntimeError: Given groups=1, weight of size [1, 3, 1], expected input[1, 1, 3] to have 3 channels, but got 1 channels instead

For Conv1d, using time series data set, I am trying to implement CNN and getting the below error:

"RuntimeError: Given groups=1, weight of size [1, 3, 1], expected input[1, 1, 3] to have 3 channels, but got 1 channels instead".

I tried to make a lot of modifications in the layers but still the error is there

Code:

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv1d(3, 1, kernel_size=1)
        self.dropout = nn.Dropout()
        self.fc1 = nn.Linear(1, 3)
        self.fc2 = nn.Linear(3, 1)
        

    def forward(self, x):
        print(x)
        x = F.relu(self.conv1(x))
        x = F.max_pool1d(x, 1)
        x = self.dropout(x)
        x = x.view(1, -1)
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
       return (x)

model.eval()
    with torch.no_grad():
        for x1,x2 in train_ldr:
            print(x1, x2)
            x1 = x1.unsqueeze(0)
            print(x1.shape)
            out = model(x1)
            print(out.shape)

And the shapes are:
tensor([[ 1.0302, -0.7415, -1.0918]]) tensor([1.])
torch.Size([1, 1, 3])

The expected input for conv1d is either (N, C_in, L_in) or (C_in, L_in) as shown in the documentation.

In your case, x1 has the shape torch.Size([1, 1, 3]) when you feed it to your model. This means that C_in=1.

However, when you defined self.conv1, the first parameter, which corresponds to C_in is 3. This is why you are getting the error.

If you want to leave your model like that, then you can simply permute the dimensions.

x1 = x1.permute(0, 2, 1)
model(x1)

However, I do not know your exact application, so you might want to change the model instead.

Hope this helps :smile:

Thank you for the solution. But if the error is:

RuntimeError: Given groups=1, weight of size [1, 3, 1], expected input[1, 1, 1] to have 3 channels, but got 1 channels instead

how do we solve the error keeping the model same. How can we change the channels in a tensor ?

Since you are leaving the model as it is, the expected C_in for self.conv1 is 3.

In this new error that you get, you can see the shape of the input you are giving to the model, which is torch.Size([1, 1, 1]). This means that N=1, C_in=1 and L_in=1.

Again, since I do not know your specific application, a possible solution would be to copy the tensor in the dimension needed.

# Random generated input
x1 = torch.rand(1, 1, 1)

# This is what you can do
x1 = x1.repeat(1, 3, 1)

# This should now work
model(x1)

If instead you want to modify the model, then you need to specify the correct number of channels C_in for the conv1.

# C_in = 1 instead of 3
self.conv1 = nn.Conv1d(1, 1, kernel_size=1)

But it all depends on how your actual data looks like and what you want to do.

Thank you for the help. I am trying to use simCLR with my model but getting a lot of dimension errors. How should I change my model to correct the dimensions. The error is in the loss function :

RuntimeError: shape '[8, 1]' is invalid for input of size -4
for epoch in range(10):
        
    print(f"Epoch [{epoch}/{epochs}]\t")
    stime = time.time()

    model.train()
    tr_loss_epoch = 0
    
    for step, (x_i, x_j) in enumerate(train_ldr):
        optimizer.zero_grad()
        x_i = x_i.unsqueeze(0)
        x_j = x_j.unsqueeze(0)
        x_j = x_j.unsqueeze(1) 
        print(x_i.shape)
        print(x_j.shape)
        # positive pair, with encoding
        x_i = x_i.permute(0, 2, 1)
        x_j = x_j.permute(0, 2, 1)
        print('Permute',x_i.shape)
        print('Permute',x_j.shape)
        z_i = model(x_i)
        x_j = x_j.repeat(1, 3, 1)
        z_j = model(x_j)
        print('z_i shape', z_i.shape)
        print('z_j shape', z_j.shape)

        loss = criterion(z_i, z_j)
        loss.backward()

        optimizer.step()
        
        if nr == 0 and step % 50 == 0:
            print(f"Step [{step}/{len(X_train)}]\t Loss: {round(loss.item(), 5)}")

        tr_loss_epoch += loss.item()

criterion = SimCLR_Loss(batch_size = 4, temperature = 0.5)
class SimCLR_Loss(nn.Module):
    def __init__(self, batch_size, temperature):
        super(SimCLR_Loss,self).__init__()
        self.batch_size = batch_size
        self.temperature = temperature

        self.mask = self.mask_correlated_samples(batch_size)
        self.criterion = nn.CrossEntropyLoss(reduction="sum")
        self.similarity_f = nn.CosineSimilarity(dim=2)

    def mask_correlated_samples(self, batch_size):
        N = 2 * batch_size
        mask = torch.ones((N, N), dtype=bool)
        mask = mask.fill_diagonal_(0)
        
        for i in range(batch_size):
            mask[i, batch_size + i] = 0
            mask[batch_size + i, i] = 0
        return mask

    def forward(self, z_i, z_j):

        N = 2 * self.batch_size

        z = torch.cat((z_i, z_j), dim=0)

        sim = self.similarity_f(z.unsqueeze(1), z.unsqueeze(0)) / self.temperature

        sim_i_j = torch.diag(sim, self.batch_size)
        sim_j_i = torch.diag(sim, -self.batch_size)
              
        positive_samples = torch.cat((sim_i_j, sim_j_i), dim=0).reshape(N, 1)
        negative_samples = sim[self.mask].reshape(N, -1)
        
        #SIMCLR
        labels = torch.from_numpy(np.array([0]*N)).reshape(-1).to(positive_samples.device).long() #.float()
        
        logits = torch.cat((positive_samples, negative_samples), dim=1)
        loss = self.criterion(logits, labels)
        loss /= N
        
        return loss

The shapes of z_i and z_j are:
z_i shape torch.Size([1, 1])
z_j shape torch.Size([1, 1])

I have not looked at it in that much detail but I have some questions.

When you are getting your data from your dataloader, what is x_i and x_j?

I assume x_i is supposed to be your data and x_j the label for this data.

If that is the case, then I do not quite understand why you also pass the label through the model?

The label is your ground truth for your loss. You might need to change its dimensions or something, but passing it through the model will give you something other than the actual label.

Sorry for the late reply.

Since I am using SimCLR model, x_i and x_j are the two data augmented representations of a sample x on which data transformations have been applied. We have to check the similarity between them and calculate the loss between them. But in my program as you said, it is considering x_j as the label x_i as the sample and thus giving different dimensions.

Oh yeah, sorry I am not that familiar with SimCLR.

However, by skimming over the paper I see that they resize the augmented images to the original image size.

So in this case x_i and x_j should have the same size before entering your model.