RuntimeError: mat1 and mat2 shapes cannot be multiplied (4x48 and 192x256)

1

I am trying to construct a Convolutional Autoencoder model for time series data, Where embed_size = 32 and time_step = 400, the model constructor is like this:

class CNNEncoder(nn.Module):
    def __init__(self, embedding_size, timestep) -> None:
        super().__init__()
        self.conv = nn.Sequential(
            nn.Conv1d(in_channels=timestep, out_channels=128, kernel_size=3, padding=1),
            nn.AvgPool1d(3),
            nn.ReLU(True),
            nn.Conv1d(in_channels=128, out_channels=64, kernel_size=3, padding=1),
            nn.AvgPool1d(3),
            nn.ReLU(True),
            nn.Conv1d(in_channels=64, out_channels=32, kernel_size=3, padding=1),  
            nn.ReLU(True),
            nn.Conv1d(in_channels=32, out_channels=16, kernel_size=3, padding=1), 
            #nn.ReLU(True),
        )

        self.fc = nn.Sequential(
            nn.Linear(192, 256),
            nn.BatchNorm1d(256),
            nn.LeakyReLU(0.2),
            nn.Linear(256, embedding_size),
        )

    def forward(self, x):
        conv = self.conv(x)
        conv = torch.flatten(conv, start_dim=1)
        return self.fc(conv)


class CNNDecoder(nn.Module):
    def __init__(self, embedding_size, timestep) -> None:
        super().__init__()
        self.fc = nn.Sequential(
            nn.Linear(embedding_size, 256),
            nn.BatchNorm1d(256),
            nn.ReLU(True),
            nn.Linear(256, 576),
            nn.ReLU(True),
            nn.Linear(576, 1024), 
            #nn.ReLU(True),
        )

        self.deconv = nn.Sequential(
            nn.ConvTranspose1d(1024, 576, 3, dilation=2),
            nn.BatchNorm1d(576),
            nn.ReLU(True),
            nn.ConvTranspose1d(576, 384, 4, padding=3, stride=3),
            nn.BatchNorm1d(384),
            nn.ReLU(True),
            nn.ConvTranspose1d(384, timestep, 3, 3, dilation=2),
            nn.Sigmoid(),
        )

    def forward(self, x):
        fc = self.fc(x)
        fc = fc.view(-1, 1024, 1) 
        return self.deconv(fc)


class AutoEncoder(nn.Module):
    def __init__(self, embed_size, time_step) -> None:
        super().__init__()

        self.encoder = CNNEncoder(embed_size, time_step)
        self.decoder = CNNDecoder(embed_size, time_step)

    def encode(self, x):
        return self.encoder(x)

    def decode(self, x):
        return self.decoder(x)

    def forward(self, x):
        z = self.encode(x)
        return self.decode(z)

But I’m getting the error:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (4x48 and 192x256)

I understand that there is a mismatch between the dimensions of the matrices I’m trying to multiply in one of the linear layers. And most probably the error occurs in CNNEncoder class, in the first linear layer of the self.fc sequential block. But I’m not sure regarding this. What would be the solution?

2

I didn’t look at your code, but if you check the complete error trace, you can identify exactly where the error happens in your code.

3

I am struggling to construct the model, need help !!!

4

The error message points to a shape mismatch in self.fc in CNNEncoder. Set the in_features of the first linear layer to 48 and it should work.

5

@ptrblck Thanks a lot, can you please explain why in_features=48 works?

6

The error message shows the activation shape first and the weight parameter shape afterwards:

I then checked if your flattening looks alright and it does:

        conv = torch.flatten(conv, start_dim=1)
        return self.fc(conv)

so the previously used in_features=192 is wrong for the current input shape and should be changed to the features in the incoming activation. Note that conv has a shape of [4, 48] based on the error message which is representing [batch_size, features] after the flattening.