Transfer learning using resnet18

vision
1

Hi,

I try to load the pretrained ResNet-18 network, create a new sequential model with the layers
of the pretrained network without the top fully connected layer and then add another fully connected layer so it would match my data (of two classes only). I tried the go by the tutorials but I keep getting the next error:
RuntimeError: size mismatch, m1: [16384 x 1], m2: [16384 x 2]

This is my cod:

class CNN(nn.Module):
    def __init__(self, num_classes):
        super(CNN, self).__init__()
              self.model_resnet = models.resnet18(pretrained=True)
              self.seq = nn.Sequential(*list(self.model_resnet.children()
              fc_inputs = self.model_resnet.fc.in_features
              self.label_model = nn.Linear(fc_inputs * 32, num_classes)

    def forward(self, images):
             with torch.no_grad():
                  features = self.seq(images)
             labels = self.label_model(features)
             return labels

Thank you!

2

I think the easier way would be to set the last fc layer in your pretrained resnet to an nn.Identity layer and pass the output to the new label_model layer.
Would this code work for you?

class CNN(nn.Module):
    def __init__(self, num_classes):
        super(CNN, self).__init__()
        self.model_resnet = models.resnet18(pretrained=True)
        fc_inputs = self.model_resnet.fc.in_features
        self.model_resnet.fc = nn.Identity()
        self.label_model = nn.Linear(fc_inputs, num_classes)

    def forward(self, images):
        with torch.no_grad():
            features = self.model_resnet(images)
        labels = self.label_model(features)
        return labels
    

model = CNN(2)
x = torch.randn(2, 3, 224, 224)
output = model(x)

I’m not sure where the fc_inputs * 32 came from.
Also, I’ve formatted your code so that I could copy it foe debugging.
If you would like to post some code, you can wrap it in three backticks ``` :wink:

3

Thank you very much for your help!
Now I try to add localization. After looking for some information on the internet, this is the code:

class CNN(nn.Module):
    def __init__(self, num_classes):
        super(CNN, self).__init__()
        self.model_resnet = models.resnet18(pretrained=True)
        fc_inputs = self.model_resnet.fc.in_features
        self.model_resnet.fc = nn.Identity()

        self.labels_model = nn.Sequential(
            nn.Linear(fc_inputs, 256),
            nn.ReLU(),
            nn.Dropout(0.4),
            nn.Linear(256, num_classes), 
            nn.LogSoftmax(dim=1) # For using NLLLoss()
        )
  
        self.conv1 = nn.Conv2d(fc_inputs, 256, kernel_size=3, padding=1)
        self.act1 = nn.ReLU()
        self.conv2 = nn.Conv2d(256, 256, kernel_size=3, padding=1)
        self.act2 = nn.ReLU()
        self.conv3 = nn.Conv2d(256, 256, kernel_size=3, padding=1)
        self.act3 = nn.ReLU()
        self.conv4 = nn.Conv2d(256, 256, kernel_size=3, padding=1)
        self.act4 = nn.ReLU()
        self.output = nn.Conv2d(256, 4, kernel_size=3, padding=1)


    def forward(self, images):
        with torch.no_grad():
            features = self.model_resnet(images)

        labels = self.labels_model(features)      
      
        box = self.conv1(features)
        box = self.act1(box)
        box = self.conv2(box)
        box = self.act2(box)
        box = self.conv3(box)
        box = self.act3(box)
        box = self.conv4(box)
        box = self.act4(box)
        box = self.output(box)
        box = out.permute(0, 2, 3, 1)
        box = out.contiguous().view(out.shape[0], -1, 4)

        return labels, box

But I get the next error:
“RuntimeError: Expected 4-dimensional input for 4-dimensional weight 256 512, but got 2-dimensional input of size [32, 512] instead”

Thank you again!

4

features will have the shape [batch_size, 512], which will throw the error if you pass it to a conv layer.
How would you like to reshape/treat this tensor?

5

I would like to get at the end a tensor of size [batch_size, 4].

6

No, I think @ptrblck’s question was how would you like the input to your conv1 be ? So, that features can be reshaped and passed in proper format.

1 Like