I’m trying to run my code using 16-nit floats. I convert the model and the data to 16-bit with no problem, but when I want to compute the loss, I get the following error:
return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
RuntimeError: “nll_loss_forward_reduce_cuda_kernel_2d_index” not implemented for ‘Half’
I’m wondering if there is something that I have to do or simply my choice of loss function does not support 16-bit floats. Thanks!