RuntimeError: mat1 and mat2 shapes cannot be multiplied (8192x1 and 2048x1000)

I want to train Cars-196 dataset in Pytorch

import torchvision.transforms as transforms
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
batch_size=4

# Load the Cars-196 train and test dataset
"""https://pytorch.org/vision/main/generated/torchvision.datasets.StanfordCars.html"""

trainset = torchvision.datasets.StanfordCars(root='./data', split='train',
                                                 download=True, transform=transform)
trainloader = DataLoader(trainset, batch_size=batch_size,
                                          shuffle=True, num_workers=4)
testset = torchvision.datasets.StanfordCars(root='./data', split='test',
                                       download=True, transform=transform)
testloader = DataLoader(testset, batch_size=batch_size,
                                         shuffle=False, num_workers=4)

I just put ResNet-50 in class and there is no modification.

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        resnet = models.resnet50(pretrained=True)
        self.features=nn.Sequential(*list(resnet.children()))
    def forward(self, x):
        x=self.features(x)
        return x
model=Model()
model=model.to(device)

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.001)

for epoch in range(10):   # Iterate data 
    model.train()
    train_loss  = 0.0
    for i, data in tqdm(enumerate(trainloader), total=len(trainloader)):
        image, labels= data
        image = image.to(device)
        labels = labels.to(device)
        # Make Gradient parameter 0...
        optimizer.zero_grad()
        
        # propagation + backpropagation + optimisation...
        outputs = model(image)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # outputs statistics
        train_loss += loss.item()
    train_loss = train_loss / len(trainset)

print('Finished Training')

But, RuntimeError: mat1 and mat2 shapes cannot be multiplied (8192x1 and 2048x1000) has been occurred.

What is the difference between training resnet-50 without class and training it using Sequential()? and how can I resolve this problem?

Wrapping the resnet modules into an nn.Sequential container will drop all functional API calls used in the original forward method such as this torch.flatten operation and will thus create this shape mismatch.
Since you are wrapping all children into nn.Sequential you could also directly use the resnet50.

Hello, thank you for your help. But, I want to modify ResNet-50 such as removing some conv layer and skip connection. In this case, should I type all the layers of ResNet in class?

Hi, when I declare self.model = models.resnet50(pretrained=True) not using nn.Sequential() it works! Thanks!

If you want to replace specific layers you could replace the internal attribute directly e.g. via:

model.fc = nn.Linear(...)

which would replace the internal .fc layer with a new nn.Linear layer while still reusing the same forward method.
For more complicated use cases you might want to create a custom nn.Module and override the forward with your custom implementation.

Hi, layer4 of ResNet-50 is consist of 3 Bottleneck containers in Sequential.
If I want to modify layer in Bottleneck container, should I make Bottleneck class separately?

It depends as mentioned in my previous post.
If you want to replace a single layer it might be easier to directly use the internal attribute and replace it. If you want to change the actual forward pass you might want to create a custom module.