Does the model.train() change the sub-modules?

Hello, I have this model class with two submodules, self.extractor and self.mlp.

class Model(nn.Module):
     def __init__(self, ):
        super(Model, self).__init__()
        self.extractor = ....
        self.extractor.eval()
        self.mlp = .....

The extractor is used to extract feature vector from the image and the self.mlp is used to perform the classification. In my case, I’d like the extractor to be in eval() mode all the time, but only train the self.mlp. However, in my training loop, I set up the Model into train():

model = Model()
def train_one_batch():
    model.train()
    ......

Now, I am confused if this line ‘model.train()’ will set the self.extractor to the train() mode and fails my goal.

Hi,
Yes, it also sets the submodules’ mode to training mode.

For the relevant code see here (search for train method in the page),
https://pytorch.org/docs/stable/_modules/torch/nn/modules/module.html#Module
where it goes through its children module

Edit: suggestion: for your problem, maybe you could still use train method and in the next line set extractor to eval mode so every submodule is to training mode except for extractor.

Thank you for the response, this really helps a lot!