How to save neuron values after activation and gradient with respect to activations using hook?

autograd
1

I need to save all the neuron values in each layer after the activation and gradient with respect to. I am very new to this and i know that i need to use hooks. Can someone please hint me how to do this for following network? Thanks.

class LeNet_5(nn.Module):

def __init__(self):
    super().__init__()

    self.conv1 = nn.Conv2d(1, 6, 5, padding=2)
    self.conv2 = nn.Conv2d(6, 16, 5)
    self.fc3 = nn.Linear(16 * 5 * 5, 120)
    self.fc4 = nn.Linear(120, 84)
    self.fc5 = nn.Linear(84, 10)

def forward(self, x):
    x = F.relu(self.conv1(x))
    x = F.max_pool2d(x, 2)
    x = F.relu(self.conv2(x))
    x = F.max_pool2d(x, 2)

    x = F.relu(self.fc3(x.view(-1, 16 * 5 * 5)))
    x = F.relu(self.fc4(x))
    x = F.log_softmax(self.fc5(x))

    return x
2

Hi,

During the forward pass. You have access to x directly if you need the value after each activation.
If you want the gradient at this point, you can get it by doing x.register_hook(your_hook_fn). Where your_hook_fn will be called with one argument: the gradient of x.