# Beginner: Should ReLU/sigmoid be called in the \_\_init\_\_ method?

**URL:** <https://discuss.pytorch.org/t/beginner-should-relu-sigmoid-be-called-in-the-init-method/18689>\
**Category:** nlp\
**Created:** [May 25, 2018, 5:52pm UTC](https://discuss.pytorch.org/t/beginner-should-relu-sigmoid-be-called-in-the-init-method/18689 "2018-05-25T17:52:07Z")\
**Posts on this page:** 1\
**Showing post:** 13

<div class="post-metadata">

**Author:** ![one\_way](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/one_way/32/15626_2.png) [@one\_way](https://discuss.pytorch.org/u/one_way)\
**Post date:** [August 28, 2019, 12:35pm UTC](https://discuss.pytorch.org/t/beginner-should-relu-sigmoid-be-called-in-the-init-method/18689/13 "2019-08-28T12:35:27Z")

</div>

Oh, Thanks!  
so the learnable parameters should be held in the \_init\_ part within MyModel. So during the forward and backward process, these parameters could be updated progressively. So in this way, MyModel can be either **written in this way** :

```python
class MyModel(nn.Module):
    def __init__ (self):
        super(MyModel, self). __init__ ()
        self.conv_like = nn.convlike() # which has its own hidden parameters 
        
    def forward(self, x):
        x = self.conv_like(x)

```

**or written in this way:**

```auto
class MyModel(nn.Module):
    def __init__ (self):
        super(MyModel, self). __init__ ()
        self.func_params = params # which will be used in nn.functional.funs 
        
    def forward(self, x):
        x = nn.functional.funs(x,self.func_params)

```

other nn.funs or nn.functional.funs which do not have hidden learnable parameters could be placed wherever you like.  
is it correct?

---

_[View the full topic](https://discuss.pytorch.org/t/beginner-should-relu-sigmoid-be-called-in-the-init-method/18689)._