How much does TORCH.AMP improve performance

mixed-precision
1

I wanted to know that does TORCH.AMP operates same as apex amp package?

I applied torch.autocast and GradScaler together on my pytorch lightning gan model but training speed did not improved and even increased slightly, gpu and memory usage decreased a bit.

my question is that are these slight changes logical or not because I red an article which improved memory consumption dramatically using amp and checkpointing.

2

Update: after some further runs I faced no improvement and the metrics event worsened. this shows that TORCH.AMP might not working properly in my code.