Rnn moel can not converge while using nn.DataParallel

my environment is win10, with torch 1.2
rnn model cannot converge while using nn.DataParallel
but it work well while not using nn.DataParallel

I using the api as the same as the following example, is there any problem with my using method in Win10, or could you give me some correct example, thanks a lot.

I encountered the same problem with PyTorch 1.4.0. At first I thought it couldn’t converge. But after a [very] * 100 long time, it started to converge [very] * 100 slowly.

And then I turned to DistributedDataParallel, and it worked. There may be a bug in LSTM + DP, I guess.

Check out how to use DDP here: https://github.com/pytorch/examples/blob/master/imagenet/main.py