Should I run update_bn() when training only a custom head while freezing the pretrained backbone?

vision
1

I’m training a model where:

  • I use a pretrained backbone (e.g., VGG, ResNet, EfficientNet).

  • I freeze the entire backbone, including all BatchNorm layers.

  • I replace the classifier head with my own custom head (which has no BatchNorm).

  • Only the head is trained; the backbone remains completely frozen.

I have two training strategies:

  1. Pick the checkpoint with lowest validation loss (standard training).

  2. Average the last 10 checkpoints (manual weight averaging, similar to mini-SWA).

In both cases, the backbone never changes.

My questions are:

  1. Should BatchNorm running statistics be updated (e.g., via BN recalibration) when the backbone is frozen?

  2. Is updating BN actually harmful in this situation since it overwrites pretrained running statistics with mismatched distributions?

  3. When averaging multiple checkpoints (only the head changes), is BN recalibration still necessary?

  4. Does requires_grad=False fully prevent BatchNorm from updating its statistics during training?

My current understanding is that BN recalibration is unnecessary—and potentially harmful—when the backbone is frozen, because the pretrained BatchNorm statistics should remain intact.

Would really appreciate clarification from the community. Thanks!