Why removing last layer is causing size mismatch?

1

Hi there,
I’m quite new to torch, so maybe it’s sth simple. I’ve trained my net, which is:

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.pool = nn.MaxPool2d(4, 4)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.fc1 = nn.Linear(44896, 4096)
        self.fc2 = nn.Linear(4096, 1024)
        self.fc3 = nn.Linear(1024, 251)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(x.size(0), 44896)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return

And now I want to remove last layer (fc3), to make a feature extractor out of my net. So I’m loading the trained model, and removing last layer:

    #loading model
    model = torch.load('./net_100_test_train')
    model.eval()

    #loading, preparing and normalizing a test image
    img=mpimg.imread('./0298_0040_0001.png')
    img = np.array(img*2-1)
    img = np.tile(img, (1, 1, 1))
    img = torch.from_numpy(img)
    img = torch.unsqueeze(img, 1)

    #removing last layer
    new_model = nn.Sequential(*list(model.children())[:-1])
    with torch.no_grad():
        output = new_model(img.to(device))

which is giving me an error:

RuntimeError: size mismatch, m1: [1952 x 93], m2: [44896 x 4096] at c:\a\w\1\s\windows\pytorch\aten\src\thc\generic/THCTensorMathBlas.cu:266

Is this because I’m using Sequential? How should I remove last layer properly, and be able to extract features?

edit:
I think I workaround it by replacing the last layer with an Identity layer, but the layer itself is still there.

1 Like
2

If you try to create a new nn.Sequential model using the layers from Net, the flatten operation will be missing.
Additionally, you are currently using only one pooling layer, as both are defined as self.pool.

Probably the most straightforward way would be to derive another class from Net and just manipulate the forward method so that it returns your desired features tensor.

However, if you would like to use the nn.Sequential approach, you could define a Flatten layer and add it to the right place:

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.pool1 = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.pool2 = nn.MaxPool2d(2, 2)
        self.fc1 = nn.Linear(44896, 4096)
        self.fc2 = nn.Linear(4096, 1024)
        self.fc3 = nn.Linear(1024, 251)

    def forward(self, x):
        x = self.pool1(F.relu(self.conv1(x)))
        x = self.pool2(F.relu(self.conv2(x)))
        print(x.shape)
        x = x.view(x.size(0), 44896)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return 


x = torch.randn(1, 1, 259, 199)
model = Net()
output = model(x)

class Flatten(nn.Module):
    def __init__(self):
        super(Flatten, self).__init__()
        
    def forward(self, x):
        return x.view(x.size(0), -1)

new_model = nn.Sequential(*[*list(model.children())[:4], Flatten(), *list(model.children())[4:-1]])
new_model(x)
1 Like
3

Thanks!
I’ll probably go with the Flatten solution.
Indeed I defined theese pools wrong, thanks for finding that! I’ve retrained the net with correct pooling and, interestingly, the accuracy has improved.

4

I am trying to remove the 4th maxpooling layer in VGG16 using

    list1 = list(vgg.classifier._modules.values())[:23]
    list1.extend(list(vgg.classifier._modules.values())[24:-1])

    list2 = list(vgg.features._modules.values())[:23]
    list2.extend(list(vgg.features._modules.values())[24:-1])
    vgg.classifier = nn.Sequential(*list1)
    self.RCNN_base = nn.Sequential(*list2)

I have also changed

self.RCNN_roi_pool = _RoIPooling(cfg.POOLING_SIZE, cfg.POOLING_SIZE, 1.0/8.0)

from

self.RCNN_roi_pool = _RoIPooling(cfg.POOLING_SIZE, cfg.POOLING_SIZE, 1.0/16.0)

However I am getting size mismatch error

RuntimeError: size mismatch, m1: [256 x 1000], m2: [4096 x 8] at /opt/conda/conda-bld/pytorch_1524577177097/work/aten/src/THC/generic/THCTensorMathBlas.cu:249

Can you please help me with this?

5

Could you post a code snippet to reproduce this issue?
I’m not sure, where and hwo self.RCNN_poi_pool is defined in your model etc.

6

I am using https://github.com/DebasmitaGhose/Multimodal_Influenza_Detection/tree/master/faster-rcnn.pytorch.

In order to remove the 4th max pooling layer I modified the lib/model/vgg16.py as follows

    list1 = list(vgg.classifier._modules.values())[:23]
    list1.extend(list(vgg.classifier._modules.values())[24:-1])

    list2 = list(vgg.features._modules.values())[:23]
    list2.extend(list(vgg.features._modules.values())[24:-1])
    vgg.classifier = nn.Sequential(*list1)
    self.RCNN_base = nn.Sequential(*list2)

Moreover in https://github.com/DebasmitaGhose/Multimodal_Influenza_Detection/blob/master/faster-rcnn.pytorch/lib/model/faster_rcnn/faster_rcnn.py, I modified the following lines

self.RCNN_roi_pool = _RoIPooling(cfg.POOLING_SIZE, cfg.POOLING_SIZE, 1.0/8.0)
self.RCNN_roi_align = RoIAlignAvg(cfg.POOLING_SIZE, cfg.POOLING_SIZE, 1.0/8.0)

In the lib/model/utils/config.py I made the following modifications

__C.DEDUP_BOXES = 1. / 16.
__C.FEAT_STRIDE = [16, ]

to

__C.DEDUP_BOXES = 1. / 8.
__C.FEAT_STRIDE = [8, ]