RuntimeError: "tanh" "_vml_cpu" not implemented for 'Half'

1

Hi All,

There is no error when I trained my code on GPU and used the dtype=torch.float16.
But when I run my code on CPU, I got the error

RuntimeError: “tanh” “_vml_cpu” not implemented for ‘Half’

I assume because torch.half isn’t supported by torch.tanh on CPU?
Is there any method can do tanh in dtype=torch.float16?

Thank you!

2

Yes, you are right and the float16 support on CPU is sparse as no speedups are expected, if I’m not mistaken. The default mixed-precision dtype on the CPU would be bfloat16.