When I use APEX to train mixed precision modes

1

I want to use GN/IN in my model.But APEX only keep BN with FP32.So what should i do when use GN/IN with FP32 in APEX？Thanks very much。

2

F.instance_norm uses batch_norm internally (line of code).
nn.GroupNorm is defined as an FP32 method here.

1 Like