My model to half

[JuanFMontesinos (Juan Montesinos)]

Hi, have a look at this post/library. Using half precision is not straight forward

GitHub - NVIDIA/apex: A PyTorch Extension: Tools for easy mixed precision...

Training partially with fp16

Hi! I was wondering if it’s possible to train partially with fp16. I’m dealing with a very big network. I would like to use fp16 in a submodule of the network. Does autograd deal with casting from float16 to float32 during the backprop? In other post i saw someone who recommends to use fp32 on batch normalization not to have convergence issues. So there should be no problem right?