Slicing input Variables and backpropagation

1

Hello all,
I have a pretrained network with a 28x28 input(MNIST) image and 10 outputs. I want to get the gradient of one of those outputs wrt the input.(how the image should change to increase the score of one digit).

If I input a single image with a variable that has requires_grad = True, I can do output[digit].backward(retain_variables=True) to do this. So, slicing the output of a network does not affect the backward prop.

However, if I input a batch of images, slice the input by x=input[k,:,:,:] and do output[k,digit].backward(retain_variables=True), x.grad remains empty(it is still None). Could you explain me why this happens and is there a way to do this without getting the grad from the whole input?

Thank you in advance!

2 Likes
2

I would slice the input and then wrap it in Variable, because autograd will only return gradients to the user for leaf nodes.

1 Like
3

Yes, but in this case I have to pass the new Variable from the network once more, right?

1 Like
4

I think I had the same confusion as the OP. Basically, I thought the slicing operation didn’t compute gradients, but x.grad (in the OP’s example) was None because intermediate variables don’t store gradients and not because slicing doesn’t compute gradients.

5

Hi, is this problem solved in v0.2?