Where to use the requires_grad=True?

In a simple linear network:
4(input)—>100
100—>3(output)
With a tensor X for the input and Y for output. Where should I use requires_grad=True in X,Y or the nn.Linear creates weights tensors that have the requires_grad=True automatically set?

Usually you don’t need to set this attribute manually.
The parameters inside the layers will be created as nn.Parameter, which will automatically have requires_grad=True.
A use case for setting this attribute manually, would be e.g. if you use some extra parameters in your model (for your fancy operations) or e.g. if you need gradients in the input (adversarial training).

1 Like