I’ve posted a similar topic and have read all topics that I found about that topic, but I just can’t seem to get it.
I’m trying to implement relevance propagation for convolutional layers.
For this, I need to calculate the gradient of a given layer with respect to its input. Since I have to calculate this gradient for intermediate layers, I do not have a scalar value at my output, but a multidimensional tensor. What I want to achieve is
The code for that would look like this:
def relprop(self, R): pself = copy.copy(self) pself.bias.data *= 0 pself.weight.data = torch.max(torch.DoubleTensor(1).zero_(), pself.weight) Z = pself.forward(self.X) + 1e-9 S = torch.div(R, Z) C = S.backward() print(self.X.grad) R = self.X * C return R
The backwards function should basically compute S * pself.weight, since I try to get the gradient with respect to X, but do this effectively.
My problem now is that I can not calculate the gradient of S, since this is only supported for scalar values. I know I can specify a gradient vector to put into backward(), but I can for the life of me not figure out what to plug in.