# Define neural network weights as torch.float16 dtype

**URL:** <https://discuss.pytorch.org/t/define-neural-network-weights-as-torch-float16-dtype/32218>\
**Category:** Uncategorized\
**Created:** [December 15, 2018, 3:34pm UTC](https://discuss.pytorch.org/t/define-neural-network-weights-as-torch-float16-dtype/32218 "2018-12-15T15:34:37Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [December 15, 2018, 3:40pm UTC](https://discuss.pytorch.org/t/define-neural-network-weights-as-torch-float16-dtype/32218/2 "2018-12-15T15:40:49Z")

</div>

Yes, it’s possible to use just parts of your model in FP16. Autograd will take care of transforming all gradients to the necessary `dtype`.  
Also, if you are using cuDNN, some layers also take mixed precision input, so that you don’t have to specifically cast them to FP16 or FP32.  
Have a look at [NVIDIA’s apex](https://github.com/NVIDIA/apex) for some mixed precision utilities as well as [this post](https://discuss.pytorch.org/t/training-partially-with-fp16/31873/2?u=ptrblck) for more information regarding apex.

---

_[View the full topic](https://discuss.pytorch.org/t/define-neural-network-weights-as-torch-float16-dtype/32218)._