The Random Seed

The initializations should yield the same random numbers, if the seed was set.
To see, if you are using cuDNN, use print(torch.backends.cudnn.enabled).