Why not use model.eval() in training_step() method on lightning

I’m wonder why we don’t use model.eval() command in training_step method of the “LightningModule”

def training_step(self, batch, batch_idx):
x, y = batch
pred = self(x) ### but our model is in training mode now …

There is two parts to this.

  • training_step is about training, so it seems natural that the model is in training mode,
  • Lightning automatically sets the model to training for training_step and to eval for validation.

Best regards

Thomas

Thanks for clarification, @tom !
Does Litghtning do the same for “on_train_epoch_end” callback or I have to set model.eval() manually?
Which of pre-defined callbacks provided with model.eval “under the hood”?

is it correct that training loss calculating using model in training mode?

I don’t know, you could find out yourself with assert not model.training (for eval mode) or so.

The rule of thumb would be that training is set up with train and validation/testing with eval, but if you want this more detailed, I would probably check with in the lightning discussions.

I think the training loss is just computed during trianing, yes.

I guess PL authors took care of switching between eval/train mode within pre defined callbacks… But problem is that when I try to predict test data in “on_fit_end” callback without using model.eval() it gives me different result than predicting outside training routine (and of course using model.eval in advance), thats why I’m wonder If ‘on_fit_end’ callback provided with model.eval … but I guess I chose inappropriate forum