# PyTorch self.module?

**URL:** <https://discuss.pytorch.org/t/pytorch-self-module/49677>\
**Category:** Uncategorized\
**Created:** [July 4, 2019, 3:10am UTC](https://discuss.pytorch.org/t/pytorch-self-module/49677 "2019-07-04T03:10:09Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![WeiQin\_Chuah](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/weiqin_chuah/32/13348_2.png) [@WeiQin\_Chuah](https://discuss.pytorch.org/u/WeiQin_Chuah)\
**Post date:** [July 4, 2019, 3:10am UTC](https://discuss.pytorch.org/t/pytorch-self-module/49677/1 "2019-07-04T03:10:09Z")

</div>

Hi, I have came across something like this, can someone explain what is the purpose of this chunk of code?

Thank you very much 🙂

```auto
for m in self.modules():
            if isinstance(m, nn.Conv2d):
                n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
                m.weight.data.normal_(0, math.sqrt(2. / n))
            elif isinstance(m, nn.Conv3d):
                n = m.kernel_size[0] * m.kernel_size[1]*m.kernel_size[2] * m.out_channels
                m.weight.data.normal_(0, math.sqrt(2. / n))
            elif isinstance(m, nn.BatchNorm2d):
                m.weight.data.fill_(1)
                m.bias.data.zero_()
            elif isinstance(m, nn.BatchNorm3d):
                m.weight.data.fill_(1)
                m.bias.data.zero_()
            elif isinstance(m, nn.Linear):
                m.bias.data.zero_()

```

---

<div class="post-metadata">

**Author:** ![shubhvachher](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shubhvachher/32/29925_2.png) [@shubhvachher](https://discuss.pytorch.org/u/shubhvachher)\
**Post date:** [July 4, 2019, 6:15am UTC](https://discuss.pytorch.org/t/pytorch-self-module/49677/2 "2019-07-04T06:15:55Z")

</div>

I’m not sure what you mean by `self.module` in your title…

The `self.modules()` method returns an iterable to the many layers or “modules” defined in the model class. This particular piece of code is using that `self.modules()` iterable to initialize the weights of the different layers present in the model. `isinstance()` checks if the particular layer “m” is an instance of a conv2d or linear or conv3d layer etc. and initializes the weights accordingly.

Why don’t you check the docs for all these functions to get a better understanding?

```auto
model.modules()
model.named_modules()
model.parameters()
model.named_parameters()
model.children()
model.named_children()

```

---

<div class="post-metadata">

**Author:** ![WeiQin\_Chuah](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/weiqin_chuah/32/13348_2.png) [@WeiQin\_Chuah](https://discuss.pytorch.org/u/WeiQin_Chuah)\
**Post date:** [July 4, 2019, 6:39am UTC](https://discuss.pytorch.org/t/pytorch-self-module/49677/3 "2019-07-04T06:39:32Z")

</div>

Thank you very much for your answer.

P.S. I am new to PyTorch and not very aware of the available resources. 😅

---

<div class="post-metadata">

**Author:** ![WeiQin\_Chuah](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/weiqin_chuah/32/13348_2.png) [@WeiQin\_Chuah](https://discuss.pytorch.org/u/WeiQin_Chuah)\
**Post date:** [July 4, 2019, 10:18pm UTC](https://discuss.pytorch.org/t/pytorch-self-module/49677/4 "2019-07-04T22:18:14Z")

</div>

@shubhvachher I have another question regarding this. Why initialize the weights manually?  
Correct me if I am wrong, the weights are initialized when you create your layers when using the `torch.nn` modules and these weights are random. Why re-initializing them?

Thank you 😄

---

<div class="post-metadata">

**Author:** ![petrell9](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/3ab097/32.png) [@petrell9](https://discuss.pytorch.org/u/petrell9)\
**Post date:** [July 5, 2019, 12:14am UTC](https://discuss.pytorch.org/t/pytorch-self-module/49677/5 "2019-07-05T00:14:15Z")

</div>

Weights are initialized to small random numbers on creation (it actually depends on the type of layer you’re using but for argument’s sake we’ll just say small random numbers), usually this isn’t a huge deal as your model will eventually learn the correct weights through gradient descent. However, depending on the model complexity this may take a long time or get stuck in a local minimum. You can set the weights manually to “guide” your model to faster convergence if you have an idea on what would work better.

---

<div class="post-metadata">

**Author:** ![shubhvachher](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shubhvachher/32/29925_2.png) [@shubhvachher](https://discuss.pytorch.org/u/shubhvachher)\
**Post date:** [July 9, 2019, 4:08am UTC](https://discuss.pytorch.org/t/pytorch-self-module/49677/6 "2019-07-09T04:08:26Z")

</div>

I’m guessing you don’t mean initializing weights with those of a pretrained model but the different types of initializations in literature.

I’m not an expert on this but I think for learning from scratch, the initializations done by pytorch internally are good enough for most tasks. You could do these initializations yourself if subject matter experts in your learning task or information you gathered from your paper review suggests some initialization techniques for that task fair better than others.

---

<div class="post-metadata">

**Author:** ![psychodrum](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/a88e57/32.png) [@psychodrum](https://discuss.pytorch.org/u/psychodrum)\
**Post date:** [July 9, 2019, 4:46am UTC](https://discuss.pytorch.org/t/pytorch-self-module/49677/7 "2019-07-09T04:46:58Z")

</div>

> [@WeiQin\_Chuah](#):
>
> iterable

You are not alone, there are no stupid questions, learning requires asking questions that are known to others , but not yourself. There was a starting point for all when we asked what a PC was. Personal Caretaker was popular prior to 90’s LOL

---

<div class="post-metadata">

**Author:** ![psychodrum](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/a88e57/32.png) [@psychodrum](https://discuss.pytorch.org/u/psychodrum)\
**Post date:** [July 9, 2019, 4:52am UTC](https://discuss.pytorch.org/t/pytorch-self-module/49677/8 "2019-07-09T04:52:29Z")

</div>

I also have to say the brilliance and help on here is incredible. I browse to see if people are being helped and treated fairly, and I must say, out of hundreds of sites this is in the top 5 for active help. Kudos people! A+

---

<div class="post-metadata">

**Author:** ![Sarah\_Salt](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sarah_salt/32/67495_2.png) [@Sarah\_Salt](https://discuss.pytorch.org/u/Sarah_Salt)\
**Post date:** [February 6, 2024, 8:06am UTC](https://discuss.pytorch.org/t/pytorch-self-module/49677/9 "2024-02-06T08:06:58Z")

</div>

Gentle reminder: when someone says they are new to the PyTorch community or NN architectures in general - be constructive not condescending with answers. I am on my 3rd year of building probabilistic NN models and I too haven’t memorized the Pytorch + Pyro handbooks. Remember, it takes courage to ask questions!

-Sarah Amiri