How to print component, layer and output size

1

Hi I’ve been trying to print out the different modules, layers and output sizes of each of them but I’ve been getting this problem:

** RuntimeError: running_mean should contain 3 elements not 64

after searching around I believe this is related to batchnorm but I’m not sure how to resolve it.

Minimal working example:

import torch
import torchvision.models as models

model = models.densenet201().cuda()
model.eval()
x = torch.FloatTensor(8, 3, 256, 256).cuda()
for feat in model.featuers:
     feat(x).size()

Any ideas?

2

You are trying to call each feature layer with the same input, i.e. x = torch.FloatTensor(8, 3, 256, 256).cuda(). That is why the second layer (BatchNorm with in_channels=64) is complaining.
If you want to call each layer one by one, you should reassign the output:

x = Variable(torch.FloatTensor(8, 3, 256, 256))
for feat in model.features:
     x = feat(x)
1 Like
3

@ptrblck Thanks a lot, you have no idea how may hours I spent on this. If you don’t mind me asking I have another question.

In the following scenario:

densenet.features.denseblock4.denselayer32(x)

I have the following error:

 self.densenet.features.denseblock4.denselayer32(x)
 *** RuntimeError: running_mean should contain 3 elements not 1888

Is that because I have to access each layer/part of the network via the forward function?

4

Oh, yeah I can imagine that. Sometimes I spend quite some time on trivial issues until I realize how complicated my approach was. :wink:

Yes, you have to call each layer before that selected one, since the activations are changing from layer to layer. In your case densenet.features.denseblock4.denselayer32 expects to have 1888 input channels.
So if you just want to call this layer, you should change the input to something like:

activation = Variable(torch.randn(1, 1888, 10, 10))
output = model.features.denseblock4.denselayer32(activation)

However, I don’t know the width and height of the activation. You could calculate it using all preceding layers or just use the for loop to get to your denselayer32 with the original input dimensions.

5

@ptrblck Once again thanks a lot for you input. I was curious about one last thing. In the above code you gave:

x = Variable(torch.FloatTensor(8, 3, 256, 256))
for feat in model.features:
     x = feat(x)

Is there a way to also access the name of each block/layer if that exists. I would expect sth like feat.name but that doesn’t exist.

I would really hope that pytorch developers really reconsider adding those features such as easily manipulating layers e.g. layer.pop, layer.push. layer.get, layer.name. That would also made the entry barrier much lower especially for anyone coming from a keras background.

6

Since features is a nn.Sequential class, you can access its modules and get the name:

for name, layer in model.features.named_modules():
    print('{}, type: {}'.format(name, type(layer)))

I’m not familiar with the model architecture, but it seems that _Denseblocks have some internal _Denselayers. So this loop will return them all:

...
denseblock4.denselayer32, type: <class 'torchvision.models.densenet._DenseLayer'>
denseblock4.denselayer32.norm.1, type: <class 'torch.nn.modules.batchnorm.BatchNorm2d'>
...

Depending on your use case you could iterate the model using its children:

for name, child in model.features.named_children():
    print('{}, type: {}'.format(name, type(child)))
    if name == 'denseblock4':
        for n, c in child.named_children():
            print(n, c)
1 Like
7

Thanks, this was my issue as well. I had changed the output dimension of one of my Conv2d layers, but forgot to change the batch norm