Best way to check if my model architecture is correct?

1

Hello folks. I am new to Pytorch and so far I like it a lot more than Tensorflow. But my question was about checking the architecture of my models. I am still getting used to the API for the layers in Pytorch, and I was hoping to find a way to visually validate the architecture of my models.

In other words, I write a stacked RNN or an LSTM with multiple layers, and I would like to know that the architecture that is coded is the architecture that I actually intended, if that makes sense. Without some explicit error on tensor shape, etc., it can be hard to make sure that I used the right argument to specify the number of hidden layers that I wanted, or the number of hidden states that I am using to plug into my fully connected layer. Especially for longer models, or models with a lot of cutting and pasting, it can be hard to ensure that the model architecture is what I intended.

Would the Tensorboard visualization fulfill this purpose, or are there other tools that take the model specification and then convert it into a visual representation of the model. I think a visual check would be the easiest way for me to tell that the model looks correct.

Any suggestions would be appreciated.
Krishna

2

torchviz could visualize the graph for a specific input, which might be helpful.
I’m not sure, how detailed e.g. the LSTM layer will be visualized, so you might try it out and see, if it helps.

Also, @tom has recently worked on a visualization package for notebooks, if I’m not mistaken, which might be helpful. :wink:

3

@ptrblck thanks for the info. I am still new to pytorch, so I don’t really know the package ecosystem yet. I will check out torchviz. That is great. Right now I am just putting the model code into pdb and then checking the sizes of the output tensors. That seems to work well–better than tensorflow, anyway :slight_smile: But it would be nice to have some visual way to visually inspect the model to make sure I did not miss anything.

Thanks again.