Backward error : 'NoneType' object has no attribute 'data'

[xavier]

hi. i want to compute the gradient with pytorch autograd in a loop (because i have may training set):

dataloader_train = DataLoader(dataset_train, batch_size=4, shuffle=True)
#b=torch.tensor([0.7],dtype=torch.float32, requires_grad=True)
b=Variable(torch.tensor([0.7]), requires_grad=True)
dataloader_train = DataLoader(dataset_train, batch_size=4, shuffle=True)
#w=torch.tensor([0.3,-0.8], dtype=torch.float32, requires_grad=True)
w=Variable(torch.tensor([0.3,-0.8]),requires_grad=True)
alpha=0.01
epochs=1

for epoch in range(epochs):
for i_batch, (x_batch,y_batch) in enumerate(dataloader_train):
print(“i_batch=”,i_batch)
for i in range(4):
w.grad.data.zero_()
b.grad.data.zero_()
print(“i_iteration=”,i)
current_train=x_batch[i,:]
print(“current_train=”,current_train)
print(“old_w=”,w)
z=torch.sum(torch.mul(current_train,w))+b
print(“z=”,z)
y_hat=torch.sigmoid(z)
#y_hat=l(w)
#print(“y_hat_vor round=”,y_hat)
#y_hat=torch.round(y_hat)
print(“y_hat after round=”,y_hat)
print(“has w grad=”,w.requires_grad)
print(“has b grad=”,b.requires_grad)
y_hat.backward()
y_hat_w_grad=w.grad.data.detach().clone()
y_hat_b_grad=b.grad.data.detach().clone()
print(“y_hat_w_grad=”,y_hat_w_grad)
print(“y_hat_b_grad=”,y_hat_b_grad)
print(“y_batch=”,y_batch[i])
w=w+(alpha*(-1)y_hat_w_grad)
b=b+(alpha(-1)*y_hat_b_grad)
print(“new_w=”,w)
print(w)

Basicaly it compute the gradient hust for the first loop.
for the second loop, it show this error :

AttributeError Traceback (most recent call last)
in
33 print(“has b grad=”,b.requires_grad)
34 y_hat.backward()
—> 35 y_hat_w_grad=w.grad.data.detach().clone()
36
37 y_hat_b_grad=b.grad.data.detach().clone()

AttributeError: ‘NoneType’ object has no attribute ‘data’

please what i am doing wrong or how can i solve it.
thanks

[xavier]

i have change the update of weights and bias like this

        w=Variable((w+(alpha*(-1)*y_hat_w_grad)),requires_grad=True)
        b=Variable((b+(alpha*(-1)*y_hat_b_grad)),requires_grad=True)

the code doesn’t not show any error but i still wonder if it compute good the gradient (with these two lines

y_hat_w_grad=w.grad.data.detach().clone()
y_hat_b_grad=b.grad.data.detach().clone()

)

[Nikronic (Nikan Doosti)]

Hi,
First of all, could you please reformat your code section using ```???
It is really hard to read.

And note that Variable has been removed from PyTorch, and now if you want to enable gradient computing on a tensor use following snippet:

@xavier

b=Variable(torch.tensor([0.7]), requires_grad=True)

b = torch.tensor([0.7])
b.requires_grad = True

But about this part:

@xavier

y_hat_w_grad=w.grad.data.detach().clone()

I think you first need to clone then detach. If you detach first, your w will no longer have grad because it is detached from computing graph.
So the solution would be something like this:

y_hat_w_grad=w.grad.data.clone().detach()

Bests
Nik

[xavier]

thank you for the info about “Variable”. what you proposed works.