We recommended to setup the mixed precision training before wrapping the model into DDP.
Wouldn’t this work for you or what the reason you would like to setup DDP before amp?
Also, note that we recommend trying out the native amp implementation using the nightly binaries, as explained here.