# Access weights of a specific module in nn.Sequential()

**URL:** <https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627>\
**Category:** Uncategorized\
**Created:** [June 1, 2017, 2:29pm UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627 "2017-06-01T14:29:18Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![mbp28](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mbp28/32/4265_2.png) [@mbp28](https://discuss.pytorch.org/u/mbp28)\
**Post date:** [June 1, 2017, 2:29pm UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/1 "2017-06-01T14:29:18Z")

</div>

Hi,

this should be a quick one, but I wasn’t able to figure it out myself.  
When I use a pre-defined module in PyTorch, I can typically access its weights fairly easily.  
However, how do I access them if I wrapped the module in nn.Sequential() first?  
Please see toy example below.

```
class My_Model_1(nn.Module):
    def __init__ (self,D_in,D_out):
        super(My_Model_1, self). __init__ ()
        self.layer = nn.Linear(D_in,D_out)
    def forward(self,x):
        out = self.layer(x)
        return out

class My_Model_2(nn.Module):
    def __init__ (self,D_in,D_out):
        super(My_Model_2, self). __init__ ()
        self.layer = nn.Sequential(nn.Linear(D_in,D_out))
    def forward(self,x):
        out = self.layer(x)
        return out

model_1 = My_Model_1(10,10)
print(model_1.layer.weight)
model_2 = My_Model_2(10,10)
# How do I print the weights now?
# model_2.layer.0.weight doesn't work.

```

Many thanks.  
Any help much appreciated.

---

<div class="post-metadata">

**Author:** ![smth](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/smth/32/13_2.png) [@smth](https://discuss.pytorch.org/u/smth)\
**Post date:** [June 1, 2017, 9:19pm UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/2 "2017-06-01T21:19:28Z")

</div>

```auto
model_2.layer[0].weight

```

---

<div class="post-metadata">

**Author:** ![mbp28](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mbp28/32/4265_2.png) [@mbp28](https://discuss.pytorch.org/u/mbp28)\
**Post date:** [June 4, 2017, 10:20am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/3 "2017-06-04T10:20:22Z")

</div>

Hi! Many thanks, this is what I was looking for. Was trying the wrong braces.

---

<div class="post-metadata">

**Author:** ![Vaijenath\_Biradar](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/v/c2a13f/32.png) [@Vaijenath\_Biradar](https://discuss.pytorch.org/u/Vaijenath_Biradar)\
**Post date:** [January 28, 2018, 7:20am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/4 "2018-01-28T07:20:10Z")

</div>

When I am doing this, the error i am getting is "The model has no attribute ‘layer’ ".

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [January 28, 2018, 2:36pm UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/5 "2018-01-28T14:36:17Z")

</div>

@Vaijenath_Biradar  
`layer` was defined in ` __init__ `:

```
self.layer = nn.Sequential(nn.Linear(D_in,D_out))

```

You have to use the variable name defined in your model.

---

<div class="post-metadata">

**Author:** ![amirbarghi](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/48db29/32.png) [@amirbarghi](https://discuss.pytorch.org/u/amirbarghi)\
**Post date:** [November 9, 2018, 5:47pm UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/6 "2018-11-09T17:47:38Z")

</div>

Hi,

Is there any way in Pytorch to get access to the layers of a model and weights in each layer without typing the layer name. Something like model.layers in keras which is discussed in the following:

> <https://stackoverflow.com/questions/50151157/keras-how-to-get-layer-index-when-already-know-layer-name>

---

<div class="post-metadata">

**Author:** ![Rick\_Sanchez](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rick_sanchez/32/10907_2.png) [@Rick\_Sanchez](https://discuss.pytorch.org/u/Rick_Sanchez)\
**Post date:** [April 10, 2019, 6:12pm UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/7 "2019-04-10T18:12:35Z")

</div>

When I using this method with model.eval() I getting different values of weights for the same example.  
Is that mean that my model don’t work correctly ?

---

<div class="post-metadata">

**Author:** ![saba](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/saba/32/19439_2.png) [@saba](https://discuss.pytorch.org/u/saba)\
**Post date:** [December 10, 2019, 7:05am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/8 "2019-12-10T07:05:43Z")

</div>

Hi ptrblck

Happy to find you here.  
I am building 2 CNN layers with 3 FC layers and using drop out two times.  
My neural network is defined as follow: Do you see any thing wrong in that? I appreciate your feedback.

import torch  
import torchvision  
import torchvision.transforms as transforms  
from torch.utils.data import TensorDataset, DataLoader  
import torch.optim as optim  
import torch.nn as nn  
from torch.utils.data.dataset import random\_split  
from torch.nn import functional as F  
import matplotlib.pyplot as plt  
from torch.autograd import Variable

class ConvNetRedo1(nn.Module):  
def **init** (self,numf1,numf2,fz1,fz2,nn2,nn3): # numf1( nnumber of filters first layer)numf2(nnumber of filters first layer)),fz1 kernel size(),fz2,nn2,nn3  
super(ConvNetRedo1, self). **init** ()  
self.numf1=numf1  
self.numf2=numf2  
self.fz1=fz1  
self.fz2=fz2  
self.nn2=nn2  
self.nn3=nn3  
self.layer1 = nn.Sequential(nn.Conv3d(1, self.numf1, kernel\_size=self.fz1, stride=1, padding=2),nn.ReLU(),nn.MaxPool3d(kernel\_size=2, stride=2))  
self.layer2 = nn.Sequential(nn.Conv3d(self.numf1,self.numf2, kernel\_size=self.fz2, stride=1, padding=2),nn.ReLU(),nn.MaxPool3d(kernel\_size=2, stride=2))  
self.fc1 = nn.Linear(3072, self.nn2) ##3027  
self.fc2 = nn.Linear( self.nn2, self.nn3) # FULLY CONNECTED LAYERS  
self.fc3 = nn.Linear( self.nn3, 1) # FULLY CONNECTED LAYERS  
self.relu = nn.ReLU() # Non-Linear ReLU Layer: max(0,x)  
self.sigmoid = nn.Sigmoid()  
self.drop\_out1 = nn.Dropout(0.5)  
self.drop\_out2 = nn.Dropout(0.5)  
self.Relu=nn.LeakyReLU(0.1, inplace=True)

```
def forward(self, x):

    x=x.unsqueeze(1).float()
    out = self.layer1(x)
    
    out = self.layer2(out)
    
    out = out.view(out.size(0), -1)
    
    out = self.fc1(out)
    out=self.drop_out1(out)
    out=self.relu(out)

    out = self.fc2(out)
    out=self.drop_out2(out)
    out=self.relu(out)
    
    out = self.fc3(out)
    out = self.sigmoid(out)
    return out
```

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [December 10, 2019, 6:03pm UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/9 "2019-12-10T18:03:49Z")

</div>

I’m not sure, if the number of features are correctly defined without knowing the input shape, but skimming through the model definition, I cannot find any obvious mistakes.  
Are you seeing any issues with the model?

---

<div class="post-metadata">

**Author:** ![saba](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/saba/32/19439_2.png) [@saba](https://discuss.pytorch.org/u/saba)\
**Post date:** [December 10, 2019, 10:29pm UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/10 "2019-12-10T22:29:56Z")

</div>

Hi  
Many thanks for your reply.  
The size of the features are good. I just want to know the way of designing, is dropout used in the good step? Is it better to use after Relu?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [December 11, 2019, 1:10am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/11 "2019-12-11T01:10:40Z")

</div>

It doesn’t matter if dropout is applied before or after the relu. I cannot see any obvious mistakes.

---

<div class="post-metadata">

**Author:** ![saba](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/saba/32/19439_2.png) [@saba](https://discuss.pytorch.org/u/saba)\
**Post date:** [December 11, 2019, 2:26am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/12 "2019-12-11T02:26:18Z")

</div>

Many thanks for your reply. Sorry I am transferring my model and data and labels to the GPU.I am not sure if I should transfer criterion and optimizer to the GPU or not?

I used them in this way

criterion = nn.BCELoss()

optimizer = torch.optim.Adam(model.parameters(), lr=.03)

I appreciate your help

Do u know any books or links which is usable?

S

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [December 11, 2019, 2:45am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/13 "2019-12-11T02:45:04Z")

</div>

You could transfer the criterion to the GPU just to avoid possible issues, but it shouldn’t be necessary for `nn.BCELoss`.  
One minor advice: I would remove the last sigmoid in your model and use `nn.BCEWithLogitsLoss` instead, as it will be numerically more stable.

Check out “Deep Learning with PyTorch” by @lantiga, @elistevens, and @tom, which can be downloaded for free on the [official website](https://pytorch.org/deep-learning-with-pytorch).  
(It’s not the full book if I’m not mistaken, as it’s still work in progress 😉 )

---

<div class="post-metadata">

**Author:** ![saba](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/saba/32/19439_2.png) [@saba](https://discuss.pytorch.org/u/saba)\
**Post date:** [December 11, 2019, 2:58am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/14 "2019-12-11T02:58:33Z")

</div>

Means finished the model to  
out = self.fc3(out)  
and use `nn.BCEWithLogitsLoss` . it has the in-built sigmoid in it?

---

<div class="post-metadata">

**Author:** ![saba](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/saba/32/19439_2.png) [@saba](https://discuss.pytorch.org/u/saba)\
**Post date:** [December 11, 2019, 3:02am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/15 "2019-12-11T03:02:52Z")

</div>

```auto
def forward(self, x):

    x=x.unsqueeze(1).float()
    out = self.layer1(x)
    
    out = self.layer2(out)
    
    out = out.view(out.size(0), -1)
    
    out = self.fc1(out)
    out=self.drop_out1(out)
    out=self.relu(out)

    out = self.fc2(out)
    out=self.drop_out2(out)
    out=self.relu(out)
    
    out = self.fc3(out)

    return out

and 

criterion=nn.BCEWithLogitsLoss() indeed the input of the this loss function should be the out put from linear layer ?
```

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [December 11, 2019, 3:05am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/16 "2019-12-11T03:05:01Z")

</div>

Yes, that’s the correct usage.

---

<div class="post-metadata">

**Author:** ![saba](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/saba/32/19439_2.png) [@saba](https://discuss.pytorch.org/u/saba)\
**Post date:** [December 11, 2019, 3:15am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/17 "2019-12-11T03:15:29Z")

</div>

Dear ptrblck

I read now the book that you suggest, I am really confused in some cases during training I saw (optimizer.zero\_grad()) is used before getting output and applying model and in this book ( lantiga) it is after getting the output?!  
which one is correct?

t\_p = model(t\_un, \*params)  
loss = loss\_fn(t\_p, t\_c)  
optimizer.zero\_grad()  
loss.backward()  
optimizer.step( )

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [December 11, 2019, 3:17am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/18 "2019-12-11T03:17:32Z")

</div>

It depends on your coding style, but it should be called before `loss.backward()` in case you don’t want explicitly to accumulate the gradients (which is a valid use case, but not the usual work flow).  
I personally try to add it right at the beginning of the training loop, since I think I remember it better, but I still forget it from time to time. 🙂

---

<div class="post-metadata">

**Author:** ![saba](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/saba/32/19439_2.png) [@saba](https://discuss.pytorch.org/u/saba)\
**Post date:** [December 11, 2019, 3:18am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/19 "2019-12-11T03:18:36Z")

</div>

I really appreciate your help 🙂

you are really helping me

---

<div class="post-metadata">

**Author:** ![saba](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/saba/32/19439_2.png) [@saba](https://discuss.pytorch.org/u/saba)\
**Post date:** [December 11, 2019, 6:20am UTC](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627/20 "2019-12-11T06:20:36Z")

</div>

Dear ptrblck

I used the thing that we discussed but the outputs are more than 1 🙂 !!

[Next page](https://discuss.pytorch.org/t/access-weights-of-a-specific-module-in-nn-sequential/3627.md?page=2)