What is mean by requires_grad(False) in resent50?

Recently I tried a tutorial. In that, they used the resnet50 from torch.models.

model = models.resnet50(pretrained=False)
for param in model.parameters():
    param.requires_grad = False
    
model.fc = nn.Sequential(nn.Linear(2048, 512),
                                 nn.ReLU(),
                                 nn.Dropout(0.2),
                                 nn.Linear(512, 256),
                                 nn.LogSoftmax(dim=1))

Can someone please explain what happens here? They are not using a pre-trained model, then what will be effect of requires_grad = False?

param.requires_grad = False will “freeze” this parameter, so that no gradient will be calculated for it.
I’m not sure, why you would want to freeze untrained parameters, but there might be specific use cases where this might be used (for experimenting?).

Yes I am using it for experimenting, could you please tell me what happens if the untrained parameters freezed?
Will it only transform the input to the shape of 2048???

Yes, resnet50 will yield a flattened activation output of [batch_size, 2048].
However, this is independent to freezing the parameters or training them.

Ohh, I am trying to classify the images which have 7 classes. But 6 samples. So I planned to do this.

  1. First to classify class-1 and rest 6 classes as one class.
  2. I used some custom affine transformations, flips, rotations, zoom on type 1, and created around 45 samples.
    Is there a better way to solve this problem?

45 samples seem to be quite low to train such a big model.
Note that resnet50 should have ~25 million parameters, if I’m not mistaken.
I doubt that you will be able to train this model successfully with 6 samples (45 with augmentation).
You would also have to create a validation dataset (and even better, another test dataset), which would further reduce the number of training samples.

Yes, that is the problem I am facing trying to solve it. Do you have any suggestion?

Depending on the use case you could create some image features using classical image processing (e.g. HOG features) and create a small random forest classifier. You could of course add augmentation, if that helps, but you would need to split the data to get some signal for validation.

1 Like

When I make it False or True?
based on dataset type!
if I use VGG16 (pre-trained on IMAGENET dataset) but my project dataset is medical images!
now! freeze parameters of model feature extractor or not?