Hello,
I am using a model from
However when I use
print(“weight[0,0] b4 average: %f” % model.classifier.weight[0,0].item())
I got the error.
Thanks,
model.classifier is defines as nn.Sequential, so you would have to index the module inside it, e.g.
model.classifier[0].weight
5 Likes
Using:
#load
pretrained_dict = torch.load(‘H:/workspace/pretrain/epoch_500_pretrain_resnet50.pth’)
model_dict = model.state_dict()
# 1. filter out unnecessary keys
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
# 2. overwrite entries in the existing state dict
model_dict.update(pretrained_dict)
# 3. load the new state dict
model.load_state_dict(model_dict)
# copy braches 0 → branches 1/2
model.branches[1].weight.copy_(model.branches[0].weight)
I get an error: AttributeError: ‘Sequential’ object has no attribute ‘weight’
Im not sure whats wrong?
Here is an image of the model structure that might reveal the issue?
It seems that this module is a Sequential of Sequentials, try
model.branches[0][1].weight
model.branches[0][1].weight.copy_(model.branches[0][0].weight)
AttributeError: ‘AdaptiveAvgPool2d’ object has no attribute ‘weight’
If I understand correctly, this adresses as follows:
model → _modules:branches → _modules: 0 → _modules:1 = AAP2d
I tried to track down where the weights are in debugging:
But I still cant find how to address it correctly, when to use the number indexing, or the . indexing?
I tried for example:
model.branches[1][0].conv1.weight .copy_( model.branches[0][0].conv1.weight )
AttributeError: ‘Sequential’ object has no attribute ‘conv1’
You don’t understand the key points.
- Use number indexing for
SequentialandModuleList - Use key for
ModuleDict - Use .module_name otherwise
For example, if I have a network like
import torch.nn as nn
class SubNet(nn.Module):
def __init__(self):
super().__init__()
self.conv = nn.Conv2d(2, 2, 2)
self.relu = nn.ReLU(inplace=True)
def forward(self, x):
return x
class Net(nn.Module):
def __init__(self):
super().__init__()
self.module = nn.Sequential(
nn.Sequential(
nn.Conv2d(1, 1, 1),
SubNet(),
),
nn.Conv2d(3, 3, 3)
)
self.conv = nn.Conv2d(4, 4, 4)
def forward(self, x):
return x
net = Net()
print(net)
The output is
Net(
(module): Sequential(
(0): Sequential(
(0): Conv2d(1, 1, kernel_size=(1, 1), stride=(1, 1))
(1): SubNet(
(conv): Conv2d(2, 2, kernel_size=(2, 2), stride=(1, 1))
(relu): ReLU(inplace=True)
)
)
(1): Conv2d(3, 3, kernel_size=(3, 3), stride=(1, 1))
)
(conv): Conv2d(4, 4, kernel_size=(4, 4), stride=(1, 1))
)
Then we have
net.module[0][0].weight → Conv2d(1, 1, 1)'s weight
net.module[0][1].conv.weight → Conv2d(2, 2, 2)'s weight
net.module[1].weight → Conv2d(3, 3, 3)'s weight
net.conv.weight → Conv2d(4, 4, 4)'s weight
Thanks you example was very informative.
I had problems understanding and seeing clearly from pycharm debug, but printing the net makes it much more clear.
What would we do without you.
1 Like
I am trying to initialise the weight but I get this error and I followed the thread but I don’t understand how I could fix in my code. I have 2 con layers and 2 fc layers
this is my architecture
#original code
class ConvNet(nn.Module):
def __init__(self):
super(ConvNet, self).__init__()
#refer the article ...
self.conv1 = nn.Sequential(
nn.Conv1d(1, 32, kernel_size=2, stride=1),
nn.Dropout(0.5),
nn.ReLU())
self.conv2 = nn.Sequential(
nn.Conv1d(32,32, kernel_size=2, stride=1),
nn.Dropout(0.5),
nn.ReLU(),
nn.MaxPool1d(2,stride=3))
self.conv3 = nn.Sequential(
nn.Conv1d(32, 32, kernel_size=2, stride=1),
nn.Dropout(0.5),
nn.ReLU())
#fully connected layers
self.fc1 = nn.Linear(32*47,32)
#self.fc2 = nn.Linear(32,1)
self.fc2 = nn.Linear(32,1)
#self.activation = nn.Softmax()
#self.activation = nn.Sigmoid()
# Initialization
nn.init.zeros_(self.conv1.weight)
nn.init.zeros_(self.conv2.bias)
def forward(self, x):
# input x :
#expected conv1d input = minibatch_size * num_channel * width
batch_size=x.size(0)
y = self.conv1(x.view(batch_size,1,-1))
y = self.conv2(y)
y = self.conv3(y)
#print(y.size())
batch_size= y.size(0)
y = y.flatten(start_dim=1)
#print(y.size())
y = self.fc1(y.view(y.size(0), -1))
#y = self.fc1(y.view(batch_size,1,-1))
y = self.fc2(y.view(batch_size,1,-1))
return y
AttributeError Traceback (most recent call last)
<ipython-input-66-063f0bb327dd> in <module>
1 #check the output size
----> 2 Model = ConvNet()
3 X = (torch.randn(32, 1, 145))
4 output=Model(X)
5 #print(output.size())
<ipython-input-65-154d3f618194> in __init__(self)
30
31 # Initialization
---> 32 nn.init.zeros_(self.conv1.weight)
33 nn.init.zeros_(self.conv2.bias)
34
~\anaconda3\lib\site-packages\torch\nn\modules\module.py in __getattr__(self, name)
1183 if name in modules:
1184 return modules[name]
-> 1185 raise AttributeError("'{}' object has no attribute '{}'".format(
1186 type(self).__name__, name))
1187
AttributeError: 'Sequential' object has no attribute 'weight'```
This line of code:
nn.init.zeros_(self.conv1.weight)
won’t work, since self.conv1 is an nn.Sequential block and thus does not have any parameters.
You would need to index the actual conv layer via self.conv1[0] and it should work:
nn.init.zeros_(self.conv1[0].weight)
1 Like