The results is different when running on different gpus!

vision
1

I use the same code and I run the code in gpu3 and gpu2 alternatively, but I got different results, I don’t know why?

image

Can someone explain why it happens?

Thanks very much.

2

I think this is normal if these are physically different GPUs since cuDNN implements several different algorithms for convolution etc. You can try to set the derministic flag for cuDNN to see if that helps. I.e., set

torch.backends.cudnn.deterministic = True

in the beginning of your code

1 Like