Running BatchNorm: "lerp_cuda" not implemented for 'Half'

I am trying to implement Running BatchNorm from here:

However, as I am on FP16, I am getting the following error:
"lerp_cuda" not implemented for 'Half'

  • Any suggestions how to get around this?

lerp is currently only dispatched for float and double as seen here. I don’t know, if half would not yield stable results or if there are other limitations.
Maybe @ngimel knows the reason.

As a workaround, you could transform your data back to a FloatTensor, apply lerp, and transform it back, if you are using a manual approach. If you are using torch.cuda.amp, you could disable autocast for this particular operation.