I added my response to your previous question Use Distributed Data Parallel correctly, let’s discuss there.