Question about requires_grad = True

1

Hello, everyone

As far as I know, model parameters (W and b) have settting requires_grad = True, by default
when they are intialized

Then, how about data matrix and label ??
Should I set requires_grad = True when I initialize data matrix (input data or batch data) as torch.tensor??

Also, should I set requires_grad = True when I input label vector into training as torch.tensor??

Thank you and have a nice day!

2

Hi,

You only need to set requires_grad=True if you need to get gradients for that Tensor.
That is why parameters are created this way by default.
You should create your inputs with this only if you plan on asking for gradients wrt to the inputs (which people usually don’t). So in general you don’t need to set it no.

1 Like
3

Dear alban
Thank you for your reply.
I understand what you mean

May I ask one more question?
What will happen if I wrongly set requires_grad = True for data matrix and label??

4

It won’t break anything, the things that can happen are:

  • you will compute gradients for some Tensors and don’t use it. Slowing down your code slightly for no reason.
  • If your label is of a non-contiguous dtype (like long), it will raise an error if you try to set requires_grad=True. Because that is only supported for contiguous dtypes.
1 Like
5

Thank you, alban
You helped me a lot

Have a nice week and see you again.

Thanks