# \[CPU\] Train network using float 16?

**URL:** <https://discuss.pytorch.org/t/cpu-train-network-using-float-16/213486>\
**Category:** Uncategorized\
**Created:** [November 26, 2024, 7:22pm UTC](https://discuss.pytorch.org/t/cpu-train-network-using-float-16/213486 "2024-11-26T19:22:22Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Aknw\_Fen](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aknw_fen/32/74156_2.png) [@Aknw\_Fen](https://discuss.pytorch.org/u/Aknw_Fen)\
**Post date:** [November 26, 2024, 7:22pm UTC](https://discuss.pytorch.org/t/cpu-train-network-using-float-16/213486/1 "2024-11-26T19:22:22Z")

</div>

I’m considering ways to improve speed and memory footprint of models and data.

Reading SO, this forum and docs, it’s still unclear to me whether it’s possible to use either float16 or bfloat16 and whether I can set this somewhere as a top level parameter.

What I have found is the setting:

```python
dtype = torch.float16
torch.set_default_dtype(dtype)

```

but this returns errors due to un-supported ops. (i’m using it with a standard neural network.)

Could anyone offer some hints?

Running `lscpu | grep dtype` shows apparently support for f16 (not bf16.)

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [November 26, 2024, 9:13pm UTC](https://discuss.pytorch.org/t/cpu-train-network-using-float-16/213486/2 "2024-11-26T21:13:26Z")

</div>

CPU workloads should support `bfloat16` in `autocast` as described in the [docs](https://pytorch.org/docs/stable/amp.html#automatic-mixed-precision-package-torch-amp):

> As shown in the CPU example section of [`torch.autocast`](https://pytorch.org/docs/stable/amp.html#torch.autocast), “automatic mixed precision training/inference” on CPU with datatype of `torch.bfloat16` only uses [`torch.autocast`](https://pytorch.org/docs/stable/amp.html#torch.autocast).

I don’t know what the status of `float16` support on CPU is and if it’s planned.