I’m trying to implement Gradient Activation mapping for segmentation problems. As a basic segmentation model, I use the U-Net (Unet_github).
To extract the activations i hooked one of the down layers (Conv->BN->ReLU)*2. In order to get the gradients of the summed outputs of the network wrt to the down layer i hooked:
torch.autograd.grad(outputs=sum(out[:,target_class, spatial_pos]), inputs=hooked_activations).cpu().data.numpy()
I’m wondering why the gradients turned out to have negative values. I checked the hooked activations and they were >=0. As i understood, the gradients should be positive as well. I changed the ReLUs within the “Down” Blocks of the Unet to inplace=False, as I red that inplace operations are dismissed by autograd.grad.
Im happy for any help!