Dropout rate of a saved model

1

Hi,

I’m just wondering how I can access the dropout rate used in a saved model.
When I save a model that has dropout layers, does the dropout rate also got saved inside the model.state_dict()? I can’t seem to find it in the dict.

2

You can just load the model and then print it. For instance:

class Model_test(nn.Module):
    def __init__(self):
        super().__init__()
        self.drop_layer = nn.Dropout(0.5)

    def forward(self, inputs):
        return self.drop_layer(inputs)
    
model = Model_test()

Now printing the model (print(model) ) returns:

Model_test(
  (drop_layer): Dropout(p=0.5, inplace=False)
)

which gives you the dropout rate you are looking for.

3

Yes, but I have the dropout rate as an input argument in my model class: init(self,drop_rate).

After training and saving the model, the next time I wish to load the model, first I need to create the model instance which actually requires me inputting the drop_rate argument. But this sets the dropout rate and then loading from the saved file does not change it.

4

Are you the one saving the state_dict? If so, you could do something like this:

torch.save({
            'model_state_dict': model.state_dict(),
            'drop_rate': drop_rate
            }, PATH)

Then load it as:

checkpoint = torch.load(PATH)
drop_rate = checkpoint['drop_rate']

model = theModelClass(drop_rate)
model.load_state_dict(checkpoint['model_state_dict'])
5

Thanks! I’ll try this.