Ａ question about a=a+b?

[Joker]

Hello , everyone , when I want implement this statement a=a+b ,I know it could be backward.

But if I want to implement this statement a[5]=a[5]+b[3] , I know this action trigger a
inplace operation because of a is a array ,and it can not be backwardin pytorch 1.2.0 , it’s memeroy has been changed I think .

And this action seems can be done in pytorch 1.1.0 but not in pytorch 1.2.0 . But if I want to implement like that , what should I write ?
Thanks for you reply and help.

Below is my wrong log .

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [212, 256, 7, 7]], which is output 0 of IndexPutBackward, is at version 3; expected version 2 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

[Nikronic (Nikan Doosti)]

Hi,

Please see below posts:

Setting part of the training value into zero

When I training my network, I want to set a specific part of latent space into the zero value based on the label. For example, after input values come out from convolution layer, there would be (B, C, H, W) 4-D tensor value would come out. I want to make half of the Channel part into zero based on the label. [image] Like this but I got an error like this. RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1,…

Why this code will throw out an inplace error

Hi, In this line, you are updating B variable but autograd needs all previous values to compute the backpropagation. You can do something like this to ensure you are not losing the value of B: import torch A = torch.nn.Parameter(torch.randn(3, 3)) B = torch.randn(2, 4, 3) z = B[0, :, :].mm(A) loss = z.sum() loss.backward() When I assign updated B to z, I am not mutating B so autograd has access to original B. Best regards Nik

Bests
Nik

[Joker]

@Nikronic Thanks for your help , friend .I solve it .

After reading your posts and pytorch doc about torch.clone() ,

clone() → Tensor 
Returns a copy of the self tensor. The copy has the same size and data type as self.

NOTE

Unlike copy_(), this function is recorded in the computation graph. Gradients propagating to the cloned tensor will propagate to the original tensor.

I changed my code to this , and it could been backward well.

 c=a.clone()
c[5] = a[5]+b[3]
loss=  m-c
loss.backward()

emmmmmmmmm if there is a function in tensor I can do it directly it will be very convenient .

Like

a[5].operation(torch.add , a[5] , b[3])
a[5].operation(torch.mm , a[5] , b[3])

And  function  `operation`  will   create  a   new   memory  to  a[5]  but  not  change  its  original  memory . So  it  could  be   backward .

Anyway , thanks for your help .