I used BN and dropout layers in my model. Do I still need to set the track_running_stats=false of BN after inference model.eval() is executed? When I execute both, the verification result is similar to the execution of one result. Does this mean that it is not necessary to set the status?
No, this argument is only used during the initialization of the batchnorm layer to define, if the running statistics should be tracked or not.
Changing this attribute later should not have any effect.
You would only have to call .train() and .eval() to switch between the different behaviors.
Recently I have found an interesting behavior (may be I am doing something wrong…) during the ResNet training.
When I apply model.eval() [with torch.no_grad()] and evaluate on validation set and switch back to model.train(), the training loss increases suddenly (gets significantly bigger than the previous training losses). But this the case for the first model.eval() only…
I was wondering if model.eval() resets BN statistics (moving mean and moving variance)?
No, that’s not the case.
Is this issue reproducible for different seeds?
Thank you so much for the reply. It is good to know that model.eval() does not change BN statistics.
As for my problem, I think I had a fundamental issue in my training. Now I am not facing the previous issue.
Good to hear the issue is solved. 
Just to be avoid a potential misunderstanding: model.eval() will use the running statistics instead of the batch statistics, but it won’t change (or reset) the running statistics.