# Whats the difference between nn.relu() vs F.relu()

**URL:** <https://discuss.pytorch.org/t/whats-the-difference-between-nn-relu-vs-f-relu/27599>\
**Category:** Uncategorized\
**Created:** [October 19, 2018, 1:21pm UTC](https://discuss.pytorch.org/t/whats-the-difference-between-nn-relu-vs-f-relu/27599 "2018-10-19T13:21:45Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Manik\_Hossain](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/manik_hossain/32/19473_2.png) [@Manik\_Hossain](https://discuss.pytorch.org/u/Manik_Hossain)\
**Post date:** [October 19, 2018, 1:21pm UTC](https://discuss.pytorch.org/t/whats-the-difference-between-nn-relu-vs-f-relu/27599/1 "2018-10-19T13:21:45Z")

</div>

whats the difference between nn.relu() vs nn.functional.relu()  
and if there no difference so why such duplication…?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [October 19, 2018, 1:28pm UTC](https://discuss.pytorch.org/t/whats-the-difference-between-nn-relu-vs-f-relu/27599/2 "2018-10-19T13:28:13Z")

</div>

`nn.ReLU()` creates an `nn.Module` which you can add e.g. to an `nn.Sequential` model.  
`nn.functional.relu` on the other side is just the functional API call to the relu function, so that you can add it e.g. in your `forward` method yourself.

Generally speaking it might depend on your coding style if you prefer modules for the activations or the functional calls. Personally I prefer the module approach if the activation has an internal state, e.g. `PReLU`.

---

<div class="post-metadata">

**Author:** ![wwiiiii](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wwiiiii/32/6613_2.png) [@wwiiiii](https://discuss.pytorch.org/u/wwiiiii)\
**Post date:** [October 19, 2018, 1:40pm UTC](https://discuss.pytorch.org/t/whats-the-difference-between-nn-relu-vs-f-relu/27599/3 "2018-10-19T13:40:41Z")

</div>

One example could be the availability of hook registration as below.

> [@How to register hook function for functional form?](https://discuss.pytorch.org/t/how-to-register-hook-function-for-functional-form/25775):
>
> I’m now trying to implement Guided backpropagation from [https://arxiv.org/abs/1412.6806](https://arxiv.org/abs/1412.6806) , which alters the gradient calculation in ReLU layer. When given model is implemented in the way containing nn.ReLU() instance as its submodule, I can easily register backward hook function to its every submodule as below. model = resnet101(pretrained=True) def hook(module, grad\_input, grad\_output): if isinstance(module, nn.ReLU): # do something return changed\_grad\_input for i, v in mod…