How to custom sigmoid activation function

1

Hello all
I am beginner in deep learning who recently researching using keras and pytorch. I want to make custom activation function that based on sigmoid with a little change like below.

new sigmoid = (1/1+exp(-x/a))

what i do in keras is like below

#CUSTOM TEMP SIGMOID
def tempsigmoid(x):
    nd=3.0 
    temp=nd/np.log(9.0) 
    return K.sigmoid(x/(temp))

i tried by making def class in pytorch but not succeed, how could i custom sigmoid in pytorch?

2

Your code should work, if you replace the numpy as Keras calls with their PyTorch equivalent:

def tempsigmoid(x):
    nd=3.0 
    temp=nd/torch.log(torch.tensor(9.0)) 
    return torch.sigmoid(x/(temp))
1 Like
3

thanks for your reply,

i tried but another error comes it is said that

temp=nd.torch.log(0.9)

log():argument ‘input’(position 1) must be Tensor, not float

i guess the calculation can not be done because i use 0.0 number as float?

4

torch.log expects a tensor, not a float value.
My code should work :wink:

1 Like
6

I tried your code and when i called the funcion in the model by

tempsigmoid(X_train)

there is an error TypeError: len() of a 0-d tensor ?

7

What shape does X_train have?
I’ve tested it with different shapes and am not sure, which function throws this error:

def tempsigmoid(x):
    nd=3.0 
    temp=nd/torch.log(torch.tensor(9.0)) 
    return torch.sigmoid(x/(temp)) 

tempsigmoid(torch.randn(1))
tempsigmoid(torch.tensor(1.))
tempsigmoid(torch.randn(1, 1))
8

thank you for your reply

my X_train data shape is ('train data shape : ', (42000, 785))

i am using reference from here kaggle and want to make some modification