Different training loss while training and while evaluating

HI. I realized my training loss is slightly different from the one that is calculated during training. Even though I added a full batch to make sure it’s not an average of the losses in each mini-batch. However, the validation loss gives the same value. I’m not using any fancy stuff such as batch normalization or so.

I had the same issue.
check this thread.
Loaded model is not the same - PyTorch Forums

Thank you. However, I’m not using fancy stuff like dropout layers. The only thing I’m using is to include gradients via torch.autograd in the cost function which are calculate inside the nn.module and they are used both in eval and train.