Could I freeze a model before loading pre-trained weights into it?

1

Hello, everyone!
If I firstly freeze a model like this:

for param in model.parameters(recurse=True):
    param.requires_grad_(False)

and then load a set of pre-trained weights into this model, will this model remains frozen? In other words, will the loaded weights refresh the “requires_grad” flag of the model params?
Thanks! :slight_smile:

2

Checking this the old fashion way suggests the model remains frozen :slight_smile:

# load a model and unfreeze its weights, for good measure
net = torch.hub.load('pytorch/vision:v0.10.0', 'resnet18', pretrained=True)
for param in net.parameters():
    param.requires_grad = True

# save the weights to disk
torch.save(net.state_dict(), "resnet18.pt")

# freeze the weights of the model
for param in net.parameters():
    param.requires_grad = False
    
# load the weights from disk into model
net.load_state_dict(torch.load("resnet18.pt"))

# count number of unfrozen weights
n_unfrozen = 0
for param in net.parameters():
    if param.requires_grad:
        n_unfrozen += 1
        
print(n_unfrozen)

Output:
0

EDIT: However, as a practical matter of code readability by a later you who may not remember the answer here for sure, it may be worth it just redundantly going through the weights after loading and asserting that they are frozen, just so it’s clear to the reader that they’re frozen.