Using NVIDIA apex for training, i cannot get same accuracy after training

distributed
2

We recommend to use the native mixed-precision training utility via torch.cuda.amp instead of apex, as it should cover more tested use cases.
More information can be found here.