I’m working with a PyTorch model from here (T2T_ViT_7).
I’m trying to freeze all layers except the last (head) layer and train it on my dataset. I want to evaluate its performance, and then unfreeze layers one by one and train it each time and see how it performs.
To initially freeze all the layers and and just unfreeze the head layer, I used:
for param in model.parameters(): param.requires_grad_(False) model.head.requires_grad_(True)
Now I want to start from the bottom, and start unfreezing layers one by one. How can I do this? Do I use model.modules() or maybe model.children()?