What will happen if I set requires_grad = False?

suppose three Linear layers A,B,C,and I set B’s requires_grad to False,can C’s loss bp to A?

2 Likes

Yes, intermediate parameters, which don’t require gradients, do not stop the backpropagation, if some earlier parameters need gradients.

1 Like

:flushed:But if the intermediate parameter don’t require gradient,how do the earlier parameters gets loss bp from the later parameters?I don’t understand the process

Does requires_grad mean compute grads but do not flush them?

The parameter gradients, which are not needed, won’t be computed (their .grad attribute won’t be updated), but the gradient calculation will continue, if it’s needed for earlier layers.

1 Like

I think you can set requires_grad=False only to leaf nodes, so is the question valid?

Yes, this 4 year old question was valid. The parameters of an intermediate modules were frozen and the question was if this would allow Autograd to properly backpropagate to the first module, which it does.

1 Like