Whats the difference between nn.relu() vs F.relu()

[wwiiiii (Jeong TaeYeong)]

One example could be the availability of hook registration as below.

How to register hook function for functional form?

I’m now trying to implement Guided backpropagation from https://arxiv.org/abs/1412.6806 , which alters the gradient calculation in ReLU layer. When given model is implemented in the way containing nn.ReLU() instance as its submodule, I can easily register backward hook function to its every submodule as below. model = resnet101(pretrained=True) def hook(module, grad_input, grad_output): if isinstance(module, nn.ReLU): # do something return changed_grad_input for i, v in mod…