Add additional layer to ResNet

vision
1

Hi @ptrblck

Before anything else, many thanks for your advice because it is always beneficial.

I want to use Resnet model with pre-trained weights, but I want to use additional input data (mean value of each image channel) besides the image at the end layer of Resnet. To do that, I plan to use a standard Resnet model, take one of its last FC layers, concatenate it with the additional input data and add FC layers processing both inputs. I changed my code as below:

if model_name == "resnet":
    class MyModel(nn.Module):
        def __init__(self):
            super(MyModel, self).__init__()
            self.model = models.resnet18(pretrained=True)
            self.model.fc = nn.Linear(self.model.fc.in_features, 515)
            # new layer
            self.fc1 = nn.Linear(515, 2)

        def forward(self, image, data):
            x1 = self.model(image)
            x2 = data
            x = torch.cat((x1, x2), dim=1)
            x = torch.nn.functional.relu(self.fc1(x))
            return x
    model = MyModel()

but while running my program I faced this error about the data loader

return self._new(self.im.resize(size, resample, box))

ValueError: height and width must be > 0

please guide me on what the error is related to?

and where I must use “param.requires_grad = True”?, because I want to use pre-trained weights in all layers to fine-tune my project.