My mistake was that I didn’t call forward() after using DDP. That’s because my model design is complicated. Thus, combining the forward processes of all modules into the same forward() function is difficult.
If using DDP without calling the DDP model’s forward() function, all parameters in the model will not be synced between multiple GPUs. This will cause serious problems.
More explanation in another discussion.