Tips or tricks on debugging a model that was packaged with nn. Sequential?

I am a self-taught python user so my knowledge is limited, I want to know how to debug my packaged model if something goes wrong in a certain operation. A basic example would be:

model = [ conv1, bn1, relu1, conv2, bn2, relu2] model = nn.Sequential( *model ) model(tensor)
Say if you run into shape error and you would like to check the input before a particular calculation how do you do it?

Right now I am trying to experiment with nn.Sequential more, with a Class function I can explicitly define and check each forward step for errors, what are some tips on debugging a packaged model?

1 Like

You could create some layers to debug the model.
If you run into a shape mismatch error, you could use a layer which prints the shape of the input:

class Print(nn.Module):
    def __init__(self):
        super(Print, self).__init__()

    def forward(self, x):
        return x

Now you could place this layer into your nn.Sequential model.


Thanks you ptrblck you are aways so active and helpful!

1 Like


I’m wondering does this also work with backward?


The Print module shouldn’t change the backward pass, as it’s just forwarding the input.
Are you seeing any issues?

No, not any issues – the graph is intact, I believe. But was just wondering if I could use the same approach to debug gradients…

To debug gradients, you could use tensor.register_hook and print out the gradient or some stats,
which might be better suited than the Print module.

1 Like

I imagine register_hook is only defined for tensors? Is it possible to achieve something similar for functional layers like relu?

Yeah, more or less.
You could use register_backward_hook on a module, but note the warning, as this might not give you the expected results for complex modules.