# Unfreeze BatchNorm only in a deep net

**URL:** <https://discuss.pytorch.org/t/unfreeze-batchnorm-only-in-a-deep-net/91177>\
**Category:** Uncategorized\
**Created:** [July 31, 2020, 3:27am UTC](https://discuss.pytorch.org/t/unfreeze-batchnorm-only-in-a-deep-net/91177 "2020-07-31T03:27:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![WeiQin\_Chuah](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/weiqin_chuah/32/13348_2.png) [@WeiQin\_Chuah](https://discuss.pytorch.org/u/WeiQin_Chuah)\
**Post date:** [July 31, 2020, 3:27am UTC](https://discuss.pytorch.org/t/unfreeze-batchnorm-only-in-a-deep-net/91177/1 "2020-07-31T03:27:59Z")

</div>

Hi,

How can I unfreeze only BatchNorm2D and freeze all the other layers is a deep net like ResNet50?

Thank you.

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 1, 2020, 7:07am UTC](https://discuss.pytorch.org/t/unfreeze-batchnorm-only-in-a-deep-net/91177/2 "2020-08-01T07:07:27Z")

</div>

You could first freeze all parameters and later unfreeze the parameters of all batchnorm layers:

```python
# Freeze all parameters
model = models.resnet50()
for param in model.parameters():
    param.requires_grad_(False)

# Unfreeze bn params
for module in model.modules():
    if isinstance(module, nn.BatchNorm2d):
        if hasattr(module, 'weight'):
            module.weight.requires_grad_(True)
        if hasattr(module, 'bias'):
            module.bias.requires_grad_(True)

# Check
out = model(torch.randn(1, 3, 224, 224))
out.mean().backward()

for name, param in model.named_parameters():
    if param.grad is not None:
        print(name, param.grad.abs().sum())

```
