Apply is not work

autograd
1

from this -> Implementation of SWISH : a self-gated activation function

Class Swish(Function):

    @staticmethod
    def forward(ctx, i):
        result = i*i.sigmoid()
        ctx.save_for_backward(result,i)
        return result

    @staticmethod
    def backward(ctx, grad_output):
        result,i = ctx.saved_variables
        sigmoid_x = i.sigmoid()
        return grad_output * (result+sigmoid_x*(1-result))

**swish= Swish.apply**

class Swish_module(nn.Module):
    def forward(self,x):
        return swish(x)
    
swish_layer = Swish_module()

Then, I received this

AttributeError                            Traceback (most recent call last)
<ipython-input-3-d633b4a504fd> in <module>()
     37         return grad_output * (result+sigm
oid_x*(1-result))
     38 
---> 39 swish= Swis
h.apply
     40 
     41 class Swish_module(nn.Module):

**AttributeError: type object 'Swish' has no attribute 'apply'**

So, I search from document then I can’t find apply method in Function.

2

Try replacing the first line:

class Swish(Function):

with:

class Swish(torch.autograd.Function):

See if this works.

3

I already do that. but I say, torch.autograd.Function also no method apply in Function class

you can see this document http://pytorch.org/docs/master/_modules/torch/autograd/function.html

1 Like
4

That’s weird. The code just works fine on my computer. Maybe upgrade your PyTorch version.

5

The Function class inherits from with_metaclass(FunctionMeta, _C._FunctionBase, _ContextMethodMixin, _HookMixin) and it is really not clear whether the super class has an apply function.

Using pytorch 0.3.0

>>> from torch.autograd import Function
>>> Function.apply
<built-in method apply of FunctionMeta object at 0x55c10dfec7f8>