Why is it not necessary to set requires_grad=True in a class subclassing nn.Module

1

Hello,

I am experimenting with pytorch and noticed that setting requires_grad=True for layers in a class subclassing nn.Module (like this Build the Neural Network — PyTorch Tutorials 2.6.0+cu124 documentation) is not necessary whereas here it is. Automatic Differentiation with torch.autograd — PyTorch Tutorials 2.6.0+cu124 documentation
Why is there this difference?

Thanks :slight_smile:

2

When you are creating a model via nn.Modules, all modules with trainable parameters will initialize nn.Parameters internally, which require gradients by design.
You can check these via print(dict(model.named_parameters()). This approach is used in the first example.

However, you don’t necessarily need to use modules and can also write your code in a pure functional way. In this case you could create trainable parameters by creating tensors with requires_grad=True yourself.