How to fully unpack the module._modules.items()

vision
1

def forward_pass_on_convolutions(self, x):
“”"
Does a forward pass on convolutions, hooks the function at given layer
“”"

    l1 = ['conv0','norm0','relu0','pool0','norm5']
    l2 = ['denseblock1','denseblock2', 'denseblock3', 'denseblock4']
    l3 = ['transition1', 'transition2', 'transition3']

    l_all = l1+l2+l3

    conv_output = None
    for module_pos, module in self.model.features._modules.items():
        print ('m', module_pos)
        if module_pos in l1:
            x = module(x)  # Forward
            if module_pos == self.target_layer:
                x.register_hook(self.save_gradient)
                conv_output = x  # Save the convolution output on that layer

        if module_pos in l2:
            for smodule_pos, smodule in module._modules.items():
                print ('sm', smodule_pos)
                x = smodule(x)  # Forward
                if smodule_pos == self.target_layer:
                    x.register_hook(self.save_gradient)
                    conv_output = x  # Save the convolution output on that layer

        if module_pos in l3:
            for smodule_pos, smodule in module._modules.items():
                print ('sm', smodule_pos)
                x = smodule(x)  # Forward
                if smodule_pos == self.target_layer:
                    x.register_hook(self.save_gradient)
                    conv_output = x  # Save the convolution output on that layer

    return conv_output, x

it gives the error:

Traceback (most recent call last):
File “main.py”, line 651, in
main(args)
File “main.py”, line 635, in main
visualize(args)
File “main.py”, line 526, in visualize
cam = grad_cam.generate_cam(prep_img, S, target_class)
File “./pytorch-cnn-visualizations/src\gradcam_.py”, line 99, in generate_cam
conv_output, model_output = self.extractor.forward_pass(input_image, cell_type)
File “./pytorch-cnn-visualizations/src\gradcam_.py”, line 71, in forward_pass
conv_output, x = self.forward_pass_on_convolutions(x)
File “./pytorch-cnn-visualizations/src\gradcam_.py”, line 49, in forward_pass_on_convolutions
x = smodule(x) # Forward
File “C:\Users\shmak\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py”, line 547, in call
result = self.forward(*input, **kwargs)
File “C:\Users\shmak\AppData\Local\Continuum\anaconda3\lib\site-packages\torchvision\models\densenet.py”, line 48, in forward
bottleneck_output = cp.checkpoint(bn_function, *prev_features)
File “C:\Users\shmak\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\utils\checkpoint.py”, line 155, in checkpoint
return CheckpointFunction.apply(function, preserve, *args)
File “C:\Users\shmak\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\utils\checkpoint.py”, line 74, in forward
outputs = run_function(*args)
File “C:\Users\shmak\AppData\Local\Continuum\anaconda3\lib\site-packages\torchvision\models\densenet.py”, line 23, in bn_function
bottleneck_output = conv(relu(norm(concated_features)))
File “C:\Users\shmak\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py”, line 547, in call
result = self.forward(*input, **kwargs)
File “C:\Users\shmak\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\batchnorm.py”, line 81, in forward
exponential_average_factor, self.eps)
File “C:\Users\shmak\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\functional.py”, line 1656, in batch_norm
training, momentum, eps, torch.backends.cudnn.enabled
RuntimeError: running_mean should contain 48 elements not 144

But if i run:

def forward_pass_on_convolutions(self, x):
    """
        Does a forward pass on convolutions, hooks the function at given layer
    """
    conv_output = None
    for module_pos, module in self.model.features._modules.items():
        print ('m', module_pos)
        x = module(x)  # Forward
        if module_pos == self.target_layer:
            x.register_hook(self.save_gradient)
            conv_output = x  # Save the convolution output on that layer
    return conv_output, x

It works fine

Also i might add, my motive is to access the denselayers modules and if possible conv1 and conv2 filters of the denselayer.

2

The error is raised, if you pass an input with the wrong number of channels to a batchnorm layer as shown here:

bn = nn.BatchNorm2d(144)
x = torch.randn(2, 48, 24, 24)
out = bn(x)
> RuntimeError: running_mean should contain 48 elements not 144

Make sure that the incoming activations has the expected number of channels as specified in the batchnorm layer initialization.

3

Hi, thank you for your reply.

But my problem is that:
when i iterate over modules like:

for module_pos, module in self.model.features._modules.items():
    x = module(x)

It does not throw any error

But when iterate like this:

for module_pos, module in self.model.features._modules.items():
    for smodule_pos, smodule in module._modules.items():
        x  = smodule(x)

It throws error, therefore there is some sub_module which is gettting skipped.

How can I fully get the lowest nested Modules (i think in a DFGS manner?)?

Thanks again for your reply.

4

I’m not sure how your model is implemented, but by iterating all modules and submodules, you will skip all functional API calls from the original forward method.
E.g. if a flatten operation was used via torch.flatten or out = out.view(out.size(0), -1), this operation won’t show up in the modules and you might end up with shape mismatch errors.

Could you check the forward implementation and search for these functional calls?

5

This is the standard pytorch implementation of memory-efficient Densenet-161 I am using.
And as I checked here, https://github.com/pytorch/vision/blob/master/torchvision/models/densenet.py

There are indeed such functional calls.
Is there are any workaround for this?

Also i think it is important to add - This is an implementation of GradCAM. Therefore it is necessary to access the submodule if its gradients are to be visualized.

6

You could add these functional calls as new nn.Modules, but I wouldn’t recommend it.

I’m not familiar with the necessary steps for GradCAM.
If you need to get gradients of parameters, you could either loop all parameters and access their param.grad attribute or register hooks for it.
Changing the forward pass shouldn’t change the gradient access, should it?

7

There is nothing from in iterating over modules, as long as you get the last nested ones properly.
The problem here is the Architecture of DenseNet where the next denselayer requires the input from all the previous layers.
Someone implementing GradCAM has to take care for that fact.