[resolved] Different outputs for the same inputs with loaded model

1

I’ve adopted this PyTorch Tutorial to experiment with using it for Named Entity Recognition (NER). It trains just fine and saves the model in a file. Now I want to copy and load this model file on a different machine. In principle, no problems as far as I can see.

My problem is now that I often get different results for the same input (a) every time a load the model again, and (b) even when the model is loaded once and I give it the identical input. Particularly (b) worries me.

I tried to search online for the problem. Here are some things I’ve tried:

  • I don’t have any dropout layer, so I cannot see any point where there is randomization involved
  • After loading the model, I do a model.eval() just to be sure
  • I’ve set torch.manual_seed(0)
  • Right now, I use a CPU machine only, so there shouldn’t be any GPU/CPU issues

Another thing I’ve tried: When - directly once the training routine is done - (1) I give the network some test data, (2) save the network, (3) re-initialize the network, (4) load the model, (5) give the network the same test data, then everything looks consistent. Only when I load the model “outside” the training, I get these different results for the same input.

At the moment, I’m test with rather small datasets, so the results are not very accurate. Still, I would assume the same output for the same input. What am I missing here?

2

Found the problem. Short answer: My stupidity.

Long answer: I did have randomness just not in the network but in the data preparation step. After loading the data into array, I first shuffled it before splitting it in training and test data. The problem here is that each time the word-to-index mapping is different. Thus before loading the state, I initialized the model each time with a different word-to-index mapping.

I now used a shuffled dataset file and skip the shuffling during the data preparation step. Now all is consistent.

2 Likes