When will pytorch introduce partial requires_grad

autograd
1

I wonder why PyTorch has not introduced partial requires_grad() to a tensor instead of making it either TRUE or FALSE for the whole tensor.

2

Hi,

There is no plan to introduce this at the moment.
What would be the use case where you require this?

3

Hi, Thanks for replying!
The partial requires_grad may stop the unnecessary gradient calculations for masked retraining of the weights!

4

Unfortunately, if you use an arbitrary mask, this won’t help in most cases, as unless your mask is very very sparse <1% nnz values, then it will be faster to do the full computation on the GPU then mask the result.