Why no eval() and train() mode switch in the DQN tutorial?

Hi, it really confuses me that why the DQN tutorial has no model.eval() and model.train(), since there are several BatchNorm layers in the model.

As I understand, model.eval() should be put before calling select_action(). And in function optimize_model(), model.train() should be put before calling the first model.forward() while model.eval() should be put before calling the second one. Is it correct?

3 Likes

I am also confused and I was going to post a similar question.

My guess is that you should run everything in train mode or only the action selection should be with model.eval(). I think training might become unstable if you use results from one model in eval and another one in train mode.