# Bfloat16 native support

**URL:** <https://discuss.pytorch.org/t/bfloat16-native-support/117155>\
**Category:** Uncategorized\
**Created:** [April 5, 2021, 7:41pm UTC](https://discuss.pytorch.org/t/bfloat16-native-support/117155 "2021-04-05T19:41:27Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![stas](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/stas/32/8409_2.png) [@stas](https://discuss.pytorch.org/u/stas)\
**Post date:** [April 7, 2021, 8:54pm UTC](https://discuss.pytorch.org/t/bfloat16-native-support/117155/5 "2021-04-07T20:54:38Z")

</div>

This approaches sounds doable, but complicated. I was hoping pytorch could provide users an API to query that, e.g. like it does with the current compute capacity of the gpu card, but I guess if CUDA doesn’t provide that API, then pytorch can’t either.

I found this compilation,

[https://machine-learning-note.readthedocs.io/en/latest/blog/cheatsheets/Nvidia\_GPU.html](https://machine-learning-note.readthedocs.io/en/latest/blog/cheatsheets/Nvidia_GPU.html)

that also includes a spreedsheet with performance numbers for the different operations. So it contains the rtx-30\* series too:

> **[Nvidia GPU](https://docs.google.com/spreadsheets/d/1AAe7gB0vfOtU_RMjE_WNcjrSmB6_LLgUIPj9cfR3KFw/edit#gid=0)**
>
> GPU summary table
> 
> Model,Series,Architecture,Processing Power (Gflops), underlined value is estimated from other value with some ratio; note that probably not all operations could be benefit from Tensor Core so usually boost might be around x2~3...

So it has TF32 numbers for Ampere cards but not bf16 yet.

---

_[View the full topic](https://discuss.pytorch.org/t/bfloat16-native-support/117155)._