Does torch.cuda.amp support O2 almost FP16 training now?

[ptrblck]

@Kungbohan

Does this mean that pytorch did not implement “O2” mixed precision? Why?

Yes, that’s correct as its deprecated apex implementation was too limiting in its flexibility and disabled a few important use cases. There are workarounds using a custom optimizer now holding the states.