Densenet Transfer Learning

1

Hi all,
I was able to implement Resnet34 using the class as follows

class Resnet34(nn.Module):
    def __init__(self, num_classes = 10):
        super(Resnet34,self).__init__()
        original_model = models.resnet34(pretrained=True)
        self.features = nn.Sequential(*list(original_model.children())[:-1])
        self.classifier = nn.Sequential(nn.Linear(512, num_classes))

    def forward(self, x):
        f = self.features(x)
        f = f.view(f.size(0), -1)
		y = self.classifier(f)
        return f,y

But when i try similar approach for densenet.

class Densenet161(nn.Module):
    def __init__(self, num_classes = 2):
        super(Densenet161,self).__init__()
        original_model = models.densenet161(pretrained=True)
        self.features = nn.Sequential(*list(original_model.children())[:-1])
        self.classifier = (nn.Linear(2208, num_classes))

    def forward(self, x):
        f = self.features(x)
        f = f.view(f.size(0), -1)
		y = self.classifier(f)
        return f,y

I get the following error

RuntimeError: size mismatch at /py/conda-bld/pytorch_1493676237139/work/torch/lib/THC/generic/THCTensorMathBlas.cu:243

I am using the same code used in Pytorch transfer learning fine tuning example.
Please suggest a method to sort the issue

2

It is best to read the model definitions first, otherwise you wont know what’s wrong.
Densenet has average pooling before the classifier stage:

Here’s a corrected version:

import torch
import torch.nn as nn
from torch.autograd import Variable
import torchvision.models as models
import torch.nn.functional as F

class Densenet161(nn.Module):
    def __init__(self, num_classes = 2):
        super(Densenet161,self).__init__()
        original_model = models.densenet161()
        self.features = nn.Sequential(*list(original_model.children())[:-1])
        self.classifier = (nn.Linear(2208, num_classes))

    def forward(self, x):
        f = self.features(x)
        f = F.relu(f, inplace=True)
        f = F.avg_pool2d(f, kernel_size=7).view(f.size(0), -1)
        y = self.classifier(f)
        return f,y


x = Densenet161()

inp = Variable(torch.randn(4, 3, 224, 224))

out = x(inp)
2 Likes
3

Thanks @smth … I downloaded the densenet code from the pytorch git and I modified the model as per my need. Its working good. Thanks

4

Here is the correct code for Transfer learning with densenet
dset_classes_number = len(class_names)
model_ft = models.densenet121(pretrained=True)
model_ft.classifier = nn.Linear(1024, dset_classes_number)

2 Likes
5

Thanks! Work perfectly!