How do you a set a seed for weight initialization in the nn module?

How do you set a seed for the random initialization of weights provided by the nn module?

1 Like

torch.manual_seed(seed)
torch.manual_seed_all(seed)

or for cuda

torch.cuda.manual_seed(seed)
torch.cuda.manual_seed_all(seed)

3 Likes

Thanks for your answer.

There is no torch.manual_seed_all(seed) when dealing with CPU (check source)

If you want to seed CPU and every GPU, you can use torch.manual_seed(seed) (see Merged Request)

1 Like

This doesn’t seem to work. I did this and initialized two LSTMs. When I printed out the parameters for each, they were different.

Have you inspected parameters when using this method to verify that two different initializations result in the same outcome?

Did you set the seed right before instantiating both models?

torch.manual_seed(seed)
modelA = MyModel()

torch.manual_seed(seed)
modelB = MyModel()
7 Likes

Nope, but doing so worked. Thanks!

1 Like