Unique module identifier

1

Is there any way to get id of module when I need that.

I can currently have a name but names may be duplicates:

for c in model.children():
    name = c._get_name()
2 
for name, moduke in model.named_modules():  # or model.named_children():
    print(name)

should return the module names, which should be unique.

Would this work for you?

3

@ptrblck : Does the inner module knows his name inside main module?
Are we destined to use this code:

for n, m in model.named_modules():
        if (m==module):

in order to grab the n (module name)
Here is the detail of using named_modules()

%matplotlib inline

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import DataLoader, Dataset, TensorDataset
from torch.optim import *
import torchvision

model = torchvision.models.segmentation.deeplabv3_resnet50(weights='DEFAULT')

## dict that will have all named modules and thair values 
## in, out, w, b (std dev and mean)
from collections import defaultdict
statDict= defaultdict(list)

''' 
has input, output and model during forward
'''
def stata(module, inp, outp):        
    for n, m in model.named_modules():
        if (m==module):           
            statDict[n+'_mean'].append(outp.mean().item())

hooks=[]
for name, m in model.named_modules(): 
    if(isinstance(m, (nn.BatchNorm2d, nn.Conv2d))):
        hooks.append(m.register_forward_hook(stata))        
        

# simplified batches

for i in range(0,3):
    x = torch.randn(2,3,80,80)
    output = model(x)

for h in hooks:
    h.remove()
4

Sorry, I don’t fully understand your code.
It seems you are trying to register a forward hook on m and are internally checking again for this module to manipulate a statDict object? What’s your use case as it seems strange to manipulate a state_dict inside a forward hook?

5

No this is statDict (statistical dictionary).
The idea is to save calculated values for each batch. Something like how the specific layer inside the model behaves when training.

6

Or to get plots like this:

image
image666×779 39.7 KB

7

I found quite the opposite can be done with this code:

from functools import reduce
model = torchvision.models.segmentation.deeplabv3_resnet50(weights='DEFAULT')

def get_module_by_name(module, access_string):
     names = access_string.split(sep='.')
     print(names)
     return reduce(getattr, names, module)

submodule = get_module_by_name(model, 'backbone.layer4.2.conv1')
print(submodule)

but this is getting the module based on his named_modules name. Can we get the named_module name easy when we have parent module and inside sub-module?

8

Ah OK, in that case your code might work, but I would probably just pass a unique identifier to the hook and store the stats using this identifier as the key to the dict as seen here.