Image Transformation inside the forward function of our network


Can we apply image transformation inside the forward function of our network?
I want to apply this transformation in the forward definition of my network.

self.transforms = transforms.Compose([transforms.ToPILImage().convert('RGB'),transforms.Resize(90),

I am using a pre-trained model(resnet50) and changing the last fully connected to my needs.

In the case when I apply transformation outside the network(transform PIL ie resize, center-crop, then Tensor) my network works.
But, for the case when I covert my input image to tensor and then inside my network try to convert it back to PIL and apply the operations. It doesn’t work.
I receive this error, even if I have applied to unsqueeze operation.

Expected 4-dimensional input for 4-dimensional weight 64 3 7 7, but got 3-dimensional input of size [3, 640, 960] instead

If I try to unsqueeze my tensor before entering the network, I get “CUDA memory full” error.

My Network Definition :-

class Network(nn.Module):
    def __init__(self):
        self.model = models.resnet50(pretrained=True)
        for param in self.model.parameters():
            param.requires_grad = False
        self.model_in_fetures = self.model.fc.in_features
        self.model.fc = nn.Sequential(nn.Linear(self.model_in_fetures,1000),nn.ReLU(),nn.Dropout(0.5),
        self.transforms = transforms.Compose([transforms.ToPILImage().convert('RGB'),transforms.Resize(90),

    def forward(self, x):
        x = self.transforms(x)
        x = x.unsqueeze(0)
        return self.model(x)

Could you try to reduce the batch size and rerun the code again?
Your current transformation will most likely not work for these issues:

  • transforms.ToPILImage() does not have a convert method, which is a method of PIL.Image
  • Transformations for PIL.Images work on single images, so you would need to loop your input tensor as:
x = torch.randn(2, 3, 224, 224)
out = []
for x_ in x:
out = torch.stack(out)