About doing double backward within DataParallel

when I am doing double backward inside a module wrapped with DataParallel, it raises RuntimeError: Scatter is not differentiable twice. So it seems we cannot do double backward inside using multiGPU. Is it true? If so, is there any alternative way to do that?

1 Like

Hi,

The only problem is that the Scatter module/function that you are using inside the DataParallel is not differentiable twice. This has not been implemented yet unfortunately.

1 Like