Calling model.eval() on during validation or just testing?

Hello everyone,

do I call model.eval(), when computing the validation loss during my training? Or do I only use evaluation mode on my test set during inference, after my training is complete?

Context: I have a train set and a test set. After each epoch, i run my test set through the model to obtain the validation loss at each epoch. Since the validation loss seems to be “simply scaled” by some orders of magnitutes (as in the curve looks the same for val and train loss, but are around 40x higher for validation), I wonder if this has anything to do with it.

Yes, you should be calling model.eval() whenever you are inferencing.

1 Like

Thank you very much.