memory in many cases for me is the primary reason. you are basically just running a bunch of forward passes through your network. if you have batchnorm layers in your network it is wise to call model.eval() before testing as this will essentially prevent that layer from messing things up if you change the batch size.
see: