Pytorch AMP is too complicated and users need to change to much code

Nvidia Apex AMP only needs a few lines of code to do amp traning, while PyTorch amp need the training codes to be changed totally. Can we simplify the API?

While the deprecated apex/amp usage might work will fewer lines of code, it lacks e.g. flexibility and is not usable for a lot of use cases e.g. when models need to be recreated etc.

Suggestions to improve the API are always welcome, so let me know if you have some! :slight_smile: