I was writing a LSTM-based model and experimented with k-fold cv, and I found that after setting all the seeds and options at the beginning of the code listed below, I still got non-deterministic results. However, I can obtain reproductible result after these settings repeated each fold. I checked the issue posted on github and I noticed that torch.backends.cudnn.deterministic is a global option, but it did not work in my code. If anyone who meets same situation, please leave a msg.
torch.backends.cudnn.deterministic = True