Differences between model.eval() and model.eval

Hy guys, I have different values in my code if I use mode.eval (That give the results almost right) instead of model.eval(). I am going to explain better.
I use my training set for testing and, cause I have a loss in training time of zero, I think that my net give me the same result of ground truth. But it doesn’t. and I note that if I use m model.eval() the results are very far from GT and if use model.eval the results are near the GT. I put my code here if I have made same mistake.


import dataset

import torch

import numpy as np
from torch.utils.data import DataLoader
from torchvision import transforms
import csv
import time
import pandas as pd




def get_test_batches():
    transform = transforms.Compose([transforms.Resize(224),
                                    transforms.ToTensor(), # conversione in tensore
                                    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])

 



    train = dataset.Dataset("./Dataset/immagini/",  "./Dataset/train.csv", transform=transform)
    # dataloader
    test_loader = DataLoader(train, batch_size=1)

    return test_loader




def predictions(model_reg, test_loader):
    since = time.time()
    print("---- START INFERENCE ----")
    print



    preds1 = []  
    preds2 = []  
    preds3 = []  
    preds4 = []  
    preds5 = []

    imgs_name = []  #

    columns=["ID1","PX", "PZ", "PU", "PV", "R"]
    df = pd.DataFrame(columns=columns)
 
    device = "cuda" if torch.cuda.is_available() else "cpu"
    model_reg.to(device)
    model_reg.eval. #WHY???
    for batch in test_loader:

        x = batch['image'].to(device)
        state = batch['start']
      


        imgs_name.append(state.item())  

        with torch.no_grad():

            pred_reg = model_reg(x)



        preds1.append(pred_reg[0][0].item())
        preds2.append(pred_reg[0][1].item())

        preds3.append(pred_reg[0][2].item())
        preds4.append(pred_reg[0][3].item())
        #print(imgs_name, preds1, preds2, preds3, preds4)

        reward = test_loader.dataset.get_reward(state.item(), pred_reg[0][0],
                                                     pred_reg[0][1],
                                                     pred_reg[0][2],
                                                     pred_reg[0][3])
        preds5.append(reward)

    df["ID1"] = imgs_name
    df["PX"] = preds1
    df["PZ"] = preds2
    df["PU"] = preds3
    df["PV"] = preds4
    df["R"]  = preds5

    df.to_csv("Predictions.csv", index=False)

    time_elapsed = time.time() - since

    print('---- INFERENCE COMPLETE in {:.0f}m {:.0f}s ----'.format(time_elapsed // 60, time_elapsed % 60))

and I call all in main loading my trained model:

resnet_reg_model = nets.resNet50_model(). #resnet with fine tuning
resnet_reg_model.load_state_dict(torch.load(“What’s the reward?.pth”, map_location=‘cpu’))
predictions(resnet_reg_model, get_test_batches())

Doing model.eval does nothing. You can remove that line and you will get the same result.
I guess this happens because your model contains batchnorm or similar layer for which the statistics are not very high quality?

Hi @albanD , thanks for your reply. I try to retrain the model, loading the model trained before. I print the input and target and I have the same results. How can possible in inference I have different values with the losses equal to zero? I think batch norm or dropout (resnet with fine tuning) don’t affect the results.
Sorry for my bad English :slight_smile:

The thing is that batchnorm behaves very differently in training and eval mode. And there are many examples of models that have this exact problem. When using the saved statistics, the quality of the result is not as good.