Random seed initialization

[Yufeng_Ma]

What’s the number of workers for your dataloader? The following post might be helpful for deterministic results.

Deterministic/non-deterministic results with PyTorch

Hi, I’ve had some trouble in reproducing results, and after reading a few posts there seem to be multiple causes for non-determinism, some which are expected. I’d like to check if I got it right. Issue 1 Dataloaders with multiple threads: They seem to be problematic, even if randoms seeds are set beforehand. [ref]. Issue 2 cuDNN: Apparently cuDNN seems to have non-deterministic kernels [ref1][ref2] Issue 3 GPU: Apparently, some reductions are non-deterministic in GPU, even without cuDNN. …