Non-deterministic result of LSTM after setting seed and option deterministic

Seth · May 15, 2018, 6:16am

I was writing a LSTM-based model and experimented with k-fold cv, and I found that after setting all the seeds and options at the beginning of the code listed below, I still got non-deterministic results. However, I can obtain reproductible result after these settings repeated each fold. I checked the issue posted on github and I noticed that torch.backends.cudnn.deterministic is a global option, but it did not work in my code. If anyone who meets same situation, please leave a msg.

random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed_all(seed)
torch.backends.cudnn.deterministic = True

erkanp · May 30, 2019, 1:15pm

I’m in the same situation, did you solve it?

Seth · May 30, 2019, 1:52pm

I did not solve this issue. Here is the doc about the randomness in pytorch.

github.com

pytorch/pytorch/blob/master/docs/source/notes/randomness.rst

Reproducibility
===============

Completely reproducible results are not guaranteed across PyTorch releases,
individual commits or different platforms. Furthermore, results need not be
reproducible between CPU and GPU executions, even when using identical seeds.

However, in order to make computations deterministic on your specific problem on
one specific platform and PyTorch release, there are a couple of steps to take.

There are two pseudorandom number generators involved in PyTorch, which you will
need to seed manually to make runs reproducible. Furthermore, you should ensure
that all other libraries your code relies on and which use random numbers also
use a fixed seed.

PyTorch
.......
You can use :meth:`torch.manual_seed()` to seed the RNG for all devices (both
CPU and CUDA)::

This file has been truncated. show original