Hi,
What’s the difference between just setting torch.manual_seed(seed),
and just setting cudnn.deterministic = True,
and setting both torch.manual_seed(seed) and cudnn.deterministic = True?
If I want a reproducible result, what is the minimum setting?
Also, here the author
says “‘You have chosen to seed training. This will turn on the CUDNN deterministic setting, ’
which can slow down your training considerably! You may see unexpected behavior when restarting from checkpoints.’”
How does that affect speed and create unexpected behavior?
Thanks!
The reproducibility docs explain the difference.
Using torch.manual_seed will seed the psuedorandom number generator, while setting cudnn.deterministic will pick deterministic algorithms in cudnn and skip potentially faster but non-deterministic ones.
1 Like