Does anyone implement class model visualization in pytorch?

I followed this paper and tried to implement the class model visualization for vgg16/alexnet. But then I failed to do so. (https://arxiv.org/abs/1312.6034). Not sure why. The attached is the fail dumbbell (image id:544) image after running 200 epochs.

The only one I know is: https://github.com/leelabcnbc/cnnvis-pytorch/blob/master/test.ipynb