Are GPU and CPU random seeds independent?

1

Hi,
I saw in one of the examples (https://github.com/pytorch/examples/blob/master/mnist/main.py) that you were setting the random seed via

torch.manual_seed(args.seed)
if args.cuda:
    torch.cuda.manual_seed(args.seed)

I couldn’t find the torch.cuda.manual_seed in the docs, and I was wondering if this is necessary to manually set the random seed or is a simple

torch.manual_seed(args.seed)

sufficient?

3 Likes
2

It is sufficient for CPU determinism, but it won’t affect the GPU PRNG state. We’ve been thinking about merging these two, and we’ll probably do so in the future.

3

Thanks! And merging them in future sounds like a good idea.

4

Agree, less chance for mysterious inconsistency for programmers who’re not aware of this.