Tips or tricks on debugging a model that was packaged with nn. Sequential?

1

I am a self-taught python user so my knowledge is limited, I want to know how to debug my packaged model if something goes wrong in a certain operation. A basic example would be:

model = [ conv1, bn1, relu1, conv2, bn2, relu2] model = nn.Sequential( *model ) model(tensor)
Say if you run into shape error and you would like to check the input before a particular calculation how do you do it?

Right now I am trying to experiment with nn.Sequential more, with a Class function I can explicitly define and check each forward step for errors, what are some tips on debugging a packaged model?

1 Like
2

You could create some layers to debug the model.
If you run into a shape mismatch error, you could use a layer which prints the shape of the input:

class Print(nn.Module):
    def __init__(self):
        super(Print, self).__init__()

    def forward(self, x):
        print(x.shape)
        return x

Now you could place this layer into your nn.Sequential model.

14 Likes
3

Thanks you ptrblck you are aways so active and helpful!

1 Like
4

Hi,

I’m wondering does this also work with backward?

Thanks,

5

The Print module shouldn’t change the backward pass, as it’s just forwarding the input.
Are you seeing any issues?

6

No, not any issues – the graph is intact, I believe. But was just wondering if I could use the same approach to debug gradients…

7

To debug gradients, you could use tensor.register_hook and print out the gradient or some stats,
which might be better suited than the Print module.

1 Like
8

I imagine register_hook is only defined for tensors? Is it possible to achieve something similar for functional layers like relu?

9

Yeah, more or less.
You could use register_backward_hook on a module, but note the warning, as this might not give you the expected results for complex modules.

2 Likes