Model (resnet34) performs better without model.eval()

7

You could use forward hook and store some stats of the intermediate activations you are interested in. This post explains how to use forward hooks on modules.