The new way of getting bad grads is the anomaly mode. But since you asked, I put up a Notebook of Adam’s bad_grad_viz adapted to modern PyTorch at https://github.com/t-vi/pytorch-tvmisc/blob/master/visualize/bad_grad_viz.ipynb.
6 Likes