Is var[0,:] unpacking for using autograd based backprop or not?

(Mohammad Mehdi Derakhshani) #1

Hello guys. According to this link, If we want to implement a loss function using autograd concept, we should not use unpacking variables. Something like this is forbidden in autograd based backprop:[0,:]

I would like to know is var[0,:] unpacking as same as[0,:]? (var is a variable type)

(Francisco Massa) #2

Indexing variables is fine. So you can do var[0, :] without a problem

(Mohammad Mehdi Derakhshani) #3

Thanks for your response, But as you have mentioned, I did following code:

output = net(images)  # batches*95*S*S
for ind in range(B):
     output[:,2+(1+coords)*ind,:,:] = torch.sqrt(output[:,2+(1+coords)*ind,:,:])
     output[:,3+(1+coords)*ind,:,:] = torch.sqrt(output[:,3+(1+coords)*ind,:,:])

But bellow error occurred:

Traceback (most recent call last):
File “”, line 200, in
File “”, line 193, in train
File “/home/mohammad/anaconda3/lib/python3.5/site-packages/torch/autograd/”, line 146, in backward
self._execution_engine.run_backward((self,), (gradient,), retain_variables)
File “/home/mohammad/anaconda3/lib/python3.5/site-packages/torch/autograd/_functions/”, line 130, in backward
i, = self.saved_tensors
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

Could you please tell me what is the problem?

(Francisco Massa) #4

Just to confirm, what is your pytorch version? (run torch.__version__ in the python interpreter).
One thing, tensor assignment is an inplace operation, so that might indicate where the problem is.

(Mohammad Mehdi Derakhshani) #5

My torch version is: 0.1.11+b13b701.

(Mohammad Mehdi Derakhshani) #6

So Is there any way to manipulate some parts of my output variable and avoid this error? Because as you can see in My loss function, I should take square of some parts of output variable!

(Francisco Massa) #7

The problem seems to be that you are trying to apply an in-place operation that can’t be in-place because the backprop wouldn’t work (sqrt).

First idea that I had was to perform the operations out of place in different tensors and then concatenate them, something like

out1 = output[:, 2::1+coords, :, :].sqrt()
out2 = output[:, 3::1+coords, :, :].sqrt()

but that still requires concatenating and shuffling around the elements in the tensor, which doesn’t sound like a good solution.

I have no better ideas now given the information I have about what you want to do, sorry

(Mohammad Mehdi Derakhshani) #8

Thanks for your recommendation. I will try that.