If I code:
w -= 0.01 * w.grad
Everything works well with no error.
However, if I code:
w = w - 0.01 * w.grad
I meet the following error:
Traceback (most recent call last):
File "/home/jihao/deep_learning/auto_gradient.py", line 34, in <module>
AttributeError: 'NoneType' object has no attribute 'zero_'
What’s the difference between the two snips of code?
w -= 0.01 * w.grad is an in-place operation, so it performs calculation on existing w and updates the value.
w = w - 0.01 * w.grad is not in-place operation, so it creates a new variable w, which does not have
requires_grad set and so the error.
You can quickly check this by calling -
print(w.requires_grad). In the first case you would get
True wheres in the second case it would be
I understand. Thank you so much!
Could you please mark the thread as solved for the benefit of other members.
Already marked. Thank you very much for solving my first question here.