I’m designing a neural network and one part of it’s a selector: depending of one value p between 0 and 1 choose between A or B.
To do so I was planning to use a natural generalization of the sigmoid function that just multiplies the input by a scalar k, that is,
Unfortunately I haven’t found something like that in the API and I was wondering if there is any reason for that? I was about to post an issue in github but I’ve decided to ask here first.
I think the sigmoid_k function is not available since, it seems it’s a one-liner:
def sigmoid_k(x, k):
return (1. / (1. + torch.exp(-k * x)))
x = torch.arange(-10, 10)
k = 2
I haven’t tested it thoroughly, so let me know, if you encounter any issues.
Yes, I already have it implemented in my code. But I was curious about why is not already implemented. I mean, the sigmoid is in the API and is also a one-liner.
But playing a bit with your code I think I might found it out. When k grows calculating the gradient may be way too much since it the function resembles the impulse function (with gradient 0 everywhere but in one point that “has infinite” gradient).
Thanks for the answer though
I assume it’s not widely used, but if you think this method should be implemented as part of the API, you could create a feature request on GitHub.
Hahaha, I’ve just written down the code without realizing the actual function. Maybe I’m just too tired.
Thanks @agadetsky for preventing more embarrassment