The acc in multi gpus is lower than one gpus

distributed
1

Hi, my code is in https://github.com/sangyx/dgl/tree/master/examples/pytorch/GATNE-T/src.

I run the main_sparse.py will get acc 0.94. But the acc will not higher than 85% with main_sparse_multi_gpu.py even I set the gpu=0.
Is there any error in my code?

My environment is Pytorch 1.6 and dgl-cu10.2 0.52. You can get the test data example in https://github.com/sangyx/dgl/tree/master/examples/pytorch/GATNE-T

2

Hey @sangyx, when using DDP, you might need to tune the batch size and learning rate a bit. See the discussion below: