RuntimeError: mat1 and mat2 shapes cannot be multiplied (8192x1 and 2048x625)

1

Hello everyone!
I’m studying CNN model, and i want to build a CNN with the following structure

CNN
Conv2D(in_channels=1, out_channels=32, kernel_size = 3, stride=1)
ReLU
MaxPool2D(kernel=2, stride=2)
Conv2D(in_channels=32, out_channels=64, kernel_size = 3, stride=1)
ReLU
MaxPool2D(kernel=2, stride=2)
Conv2D(in_channels=64, out_channels=128, kernel_size = 3, stride=1)
ReLU
MaxPool2D(kernel=2, stride=2)
Linear(in_features=2048, out_features=625)
ReLU
Dropout
Linear(in_features=625, out_features=10)

And then, the script i wrote is:

class CNN(nn.Module):
def init(self):
super(CNN, self).init()
self.conv1 = nn.Conv2d(in_channels=1, out_channels=32, kernel_size=3, stride=1)
self.conv2 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, stride=1)
self.conv3 = nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, stride=1)

    self.dropout1 = nn.Dropout(0.25)
    self.dropout2 = nn.Dropout(0.5)

    self.fc1 = nn.Linear(in_features=2048, out_features=625)
    self.fc2 = nn.Linear(in_features=625, out_features=10)

def forward(self, x):
    x = self.conv1(x)
    x = F.relu(x)
    x = F.max_pool2d(x, 2)

    x = self.conv2(x)
    x = F.relu(x)
    x = F.max_pool2d(x, 2)

    x = self.conv3(x)
    x = F.relu(x)
    x = F.max_pool2d(x, 2)
    x = self.fc1(x)
    x = F.relu(x)
    x = self.dropout1(x)
    x = self.fc2(x)
    output = F.log_softmax(x, dim=1)
    return output

but i get RuntimeError!
RuntimeError: mat1 and mat2 shapes cannot be multiplied (8192x1 and 2048x625)

I would really appreciate it if you could let me know what’s wrong with my example.

2

Hi
what is the shape of your input?
you are likely getting this error because the shape of x when applying fc1 does not match.
you probably need to Flatten your tensor and check if its dimension is [batch_size,2048],or maybe you need to change the value of stride or maybe add some maxpool layers to make it match

3

Thanks! nassim!
I use MNIST dataset, so the shape of input is [1, 1, 28, 28]
So is in_features=2048 an invalid value in Linear(in_features=2048, out_features=625)?

4

Yes in_features=2048 is an invalid value in this case.
You start with x of shape [1,1,28,28] after your first conv layer, x will be of shape [1,32,26,26],
after the max_pool layer ,the shape is [1,32,13,13],after conv2 it is [1,64,11,11],after the next max_pool it is [1,64,5,5], after conv3 it is [1,128,3,3],then after the last max_pool[1,128,1,1], so your fc1 need to be of type (in_features=128,out_features=x), but before applying a fully connected layer your data needs to be of shape [batch_size,in_features] so you need to transform your data from [x,128,1,1] to [x,128].
You can do that by adding x=torch.squeeze(x) before applying fc1.