How to fix parameter in CNN (vgg19)

1

I use VGG19 for train model but I’m curious that I should use which one and what is different

  1. model.features.eval()
  2. for param in model.features.parameters():
    param.requires_grad = False
  3. with torch.no_grad():

example

# fix the weight of convolution layers
model.features.eval()

# modify classifier
model.classifier = torch.nn.Sequential(
  nn.Linear(25088, 4096),
  nn.ReLU(inplace=True),
  nn.Dropout(p=0.5, inplace=False),
  nn.Linear(4096, 4096),
  nn.ReLU(inplace=True),
  nn.Dropout(p=0.5, inplace=False),
  torch.nn.Linear(4096, 2))
2

  1. This sets the model in evaluation mode, which changes the behavior of some layers but does not freeze trainable parameters. After calling model.eval() e.g. dropout layers will be disabled and batchnorm layers will use their running stats to normalize the input activation.

  2. These lines of code will freeze all properly registered trainable parameters of the .features submodule.

  3. This guard will disable gradient calculation and will not store intermediate forward activations thus reducing the memory usage.

1 Like