Optimizer params not in the optimizer state dict

These numbers represent the ids of the parameters you’ve passed to the optimizer.
E.g. if your model has a linear layer called fc1, you can check the id using:

print(id(model.fc1.weight))

and should find this id also in the optimizer’s state_dict.

4 Likes