You could register forward hooks to the layers (as described here) and try to visualize the feature maps.