Torch autocast's gradient

[gusye1234 (Jianbai Ye)]

So is there an FP16 version of gradient during training? or FP32 all the time.