A little confusing that both exist. Is any of these deprecated?
we should now deprecate torch.nn.functional.tanh on master, as tensors and variables are now merged.
If I use nn.Tanh I have to declare it in my_model.init(), e.g.
self.tanh = nn.Tanh()
Whereas I can use nn.functional.tanh directly in my_model.forward(), e.g.
output = nn.functional.tanh(input)
If you deprecate nn.functional.tanh I could do
output = nn.Tanh()(input)
where ever I need the functional form, but it would be slower because of the class instantiation.
What has this to do with Variables and Tensors? I am confused.
Or you could still use torch.tanh():
output = input.tanh()
I am expecting output having both positive and negative values. So,
output = torch.tanh(model(input))
in the final output of network should be fine?
Or are any other variants available?
There are only few activations like torch.tanh in torch.
But for torch.nn.functional , i found alot of functional activations.
Why is that?
Same confusion here