When I use APEX to train mixed precision modes

I want to use GN/IN in my model.But APEX only keep BN with FP32.So what should i do when use GN/IN with FP32 in APEX?Thanks very much。

F.instance_norm uses batch_norm internally (line of code).
nn.GroupNorm is defined as an FP32 method here.

1 Like