Which one works faster? a) or b)

Intel_Novel · February 27, 2019, 3:27pm

tensor = torch.randn((1, 2), requires_grad=True)

print(torch.tanh(tensor)) #a)
print(nn.Tanh()(tensor)) #b)

#nn.functional.tanh is deprecated
#print(torch.nn.functional.tanh(tensor))

vabh · February 27, 2019, 3:32pm

They are actually the same, and so should be identical it terms of speed
Is you look at the source, nn.Tanh uses torch.tanh in its forward method.

github.com

pytorch/pytorch/blob/master/torch/nn/modules/activation.py#L299


.. image:: scripts/activation_images/Tanh.png


Examples::


    >>> m = nn.Tanh()
    >>> input = torch.randn(2)
    >>> output = m(input)
"""


@weak_script_method
def forward(self, input):
    return torch.tanh(input)




@weak_module
class ELU(Module):
r"""Applies the element-wise function:


.. math::
    \text{ELU}(x) = \max(0,x) + \min(0, \alpha * (\exp(x) - 1))

Intel_Novel · February 27, 2019, 3:33pm

I saw this just a moment ago. Thanks, I would say torch.tanh(tensor) is just a tiny bit faster.

Intel_Novel · February 27, 2019, 3:35pm

BTW, do you have any idea what is @weak_script_method suppose to do?