Optimizing losses on different GPUs

Your problem seems to be the same as this.