Hi, I have a similar issue which I posted here: Can't achive reproducability / determinism in pytorch training - #8 by Ecem_sogancioglu
I use FasterRCNN PyTorch implementation, I updated PyTorch to nightly release and set torch.use_deterministic_algorithms(True). I also set the environmental variable CUBLAS_WORKSPACE_CONFIG=:16:8, however, the results are still not reproducible. When trained on the CPU, the results are reproducible.