when I am doing double backward inside a module wrapped with DataParallel
, it raises RuntimeError: Scatter is not differentiable twice
. So it seems we cannot do double backward inside using multiGPU. Is it true? If so, is there any alternative way to do that?
1 Like
Hi,
The only problem is that the Scatter
module/function that you are using inside the DataParallel
is not differentiable twice. This has not been implemented yet unfortunately.
1 Like