I’d say it is a clear: No. You would keep the model in training state (model.train()) while training and only set it to model.eval() during evaluation (e.g. validation interleaved with training epochs, inference proper). Don’t for get to set it back to training state for the next epoch if you do.
I must admit I have trouble to see why the person helping you on stackoverflow would come to a different conclusion.
The other way round it would make more sense btw., for things like test time dropout, you would actually avoid setting the dropout layers to evaluation mode.
P.S.: I would advise against crossposting. If you get the same answer, it’ll be wasted time for people answering with no benefit to you. If you get different ones, you need to make up your mind which would have been the better forum to ask in anyways and it’ll be wasted time for people answering.
Thanks for quick reply.
I do not get the point why a dropout layer should not to be set t eval mode. If it is not set to eval mode then the output for the same example can be different for different runs.
PS: as for cross-posting I think it’s not necessarily bad as the user base for different forums can be quite different. And its not necessarily my question on other forums.