Hello! I have this simple (part of a) code:

``````a = torch.tensor(0.5)
lr = 1e-3
for i in range(20):
yy = torch.cos(a*x)
loss = torch.mean((yy-y)**2)
loss.backward()
``````

Where x and y are fixed numbers, generated using y=(0.538*x). Basically I want to make a go from 0.5 to 0.538 using gradient descent.

The first loop works fine and a goes in the right direction, but at the second loop I am getting this error at the `a = a-a.grad.data` line: `AttributeError: 'NoneType' object has no attribute 'data'`. What am I doing wrong? What changes from the first to second iteration? Thank you!

Hi,

Few things:

• Do not use `.data`, it’s not a thing anymore and causes a lot of bugs. If you want to make a weight update here, do :
``````with torch.no_grad():
``````
• And your real issue is that when you do `a = xxx`. Then you re-assign `a` in a differentiable manner. Meaning that your new `a` is not a leaf anymore. Using the code in the point above will change `a` inplace and it will thus remain a leaf.
• (EDIT) Also `a.grad.zero_()` inside the no_grad block as @crowsonkb mentioned below

``````a = torch.tensor(0.5)
lr = 1e-3
for i in range(20):
yy = torch.cos(a*x)
loss = torch.mean((yy-y)**2)
loss.backward()
``````

The following also works (but may be less efficient?) :

``````    with torch.no_grad():
a = a - lr * a.grad
The problem you are running into is that when you do `a = a - a.grad` you are getting a new `a` which does not have `requires_grad` set. So either you need to do in-place operations on `a` (preserving the flag) or manually re-enable it. Also, the gradient descent step has to be done inside a `no_grad()` context.
I covers exactly the problem you were having and goes a bit more into detail about what is happening. 