Apply is not work

from this -> Implementation of SWISH : a self-gated activation function

Class Swish(Function):

    def forward(ctx, i):
        result = i*i.sigmoid()
        return result

    def backward(ctx, grad_output):
        result,i = ctx.saved_variables
        sigmoid_x = i.sigmoid()
        return grad_output * (result+sigmoid_x*(1-result))

**swish= Swish.apply**

class Swish_module(nn.Module):
    def forward(self,x):
        return swish(x)
swish_layer = Swish_module()

Then, I received this

AttributeError                            Traceback (most recent call last)
<ipython-input-3-d633b4a504fd> in <module>()
     37         return grad_output * (result+sigm
---> 39 swish= Swis
     41 class Swish_module(nn.Module):

**AttributeError: type object 'Swish' has no attribute 'apply'**

So, I search from document then I can’t find apply method in Function.

Try replacing the first line:

class Swish(Function):


class Swish(torch.autograd.Function):

See if this works.

I already do that. but I say, torch.autograd.Function also no method apply in Function class

you can see this document

1 Like

That’s weird. The code just works fine on my computer. Maybe upgrade your PyTorch version.

The Function class inherits from with_metaclass(FunctionMeta, _C._FunctionBase, _ContextMethodMixin, _HookMixin) and it is really not clear whether the super class has an apply function.

Using pytorch 0.3.0

>>> from torch.autograd import Function
>>> Function.apply
<built-in method apply of FunctionMeta object at 0x55c10dfec7f8>