# FP8 support on H100 **URL:** \ **Category:** mixed-precision\ **Created:** [October 18, 2023, 8:15pm UTC](https://discuss.pytorch.org/t/fp8-support-on-h100/190236 "2023-10-18T20:15:20Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![navmarri](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/navmarri/32/38412_2.png) [@navmarri](https://discuss.pytorch.org/u/navmarri)\ **Post date:** [October 18, 2023, 8:15pm UTC](https://discuss.pytorch.org/t/fp8-support-on-h100/190236/1 "2023-10-18T20:15:21Z")
with H100 supporting FP8, is there any plan to support FP8 training in pytorch? Looks like the only alternative is to use FP8 through [nvidia transformer engine](https://github.com/NVIDIA/TransformerEngine). Even transformer engine doesn’t support all nn modules like conv layers for FP8 support. ---
**Author:** ![marksaroufim](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/marksaroufim/32/36747_2.png) [@marksaroufim](https://discuss.pytorch.org/u/marksaroufim)\ **Post date:** [October 18, 2023, 8:44pm UTC](https://discuss.pytorch.org/t/fp8-support-on-h100/190236/2 "2023-10-18T20:44:43Z")
@drisspg has been working on this ---
**Author:** ![navmarri](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/navmarri/32/38412_2.png) [@navmarri](https://discuss.pytorch.org/u/navmarri)\ **Post date:** [October 18, 2023, 10:02pm UTC](https://discuss.pytorch.org/t/fp8-support-on-h100/190236/3 "2023-10-18T22:02:58Z")
@marksaroufim is there a timeline for the feature to be available? ---
**Author:** ![drisspg](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/drisspg/32/48977_2.png) [@drisspg](https://discuss.pytorch.org/u/drisspg)\ **Post date:** [October 19, 2023, 11:13pm UTC](https://discuss.pytorch.org/t/fp8-support-on-h100/190236/4 "2023-10-19T23:13:15Z")
Hey @navmarri we actually have landed some fp8 primitives already in PyTorch. Specifically we have landed fp8e4m3 and fp8e5m2 datatypes and a private function (with no BC/FC guarantees) for doing scaled matmul on h100 machines. For a quick example script of how you can use it, check out: > ---
**Author:** ![navmarri](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/navmarri/32/38412_2.png) [@navmarri](https://discuss.pytorch.org/u/navmarri)\ **Post date:** [October 20, 2023, 12:05am UTC](https://discuss.pytorch.org/t/fp8-support-on-h100/190236/5 "2023-10-20T00:05:44Z")
@drisspg awesome! does this work with FP8 precision training using FSDP on H100s? ---
**Author:** ![KohakuBlueleaf](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kohakublueleaf/32/64874_2.png) [@KohakuBlueleaf](https://discuss.pytorch.org/u/KohakuBlueleaf)\ **Post date:** [October 23, 2023, 8:30am UTC](https://discuss.pytorch.org/t/fp8-support-on-h100/190236/6 "2023-10-23T08:30:33Z")
As I know (and also be mentioned in Nvidia/TransformerEngine) RTX 4000 series (compute capibility 8.9) should also support FP8 computing. Does pytorch support that too? or for now it is only for H100 and need further works for sm8.9 cards? ---
**Author:** ![bhack](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bhack/32/62677_2.png) [@bhack](https://discuss.pytorch.org/u/bhack)\ **Post date:** [March 8, 2024, 5:18pm UTC](https://discuss.pytorch.org/t/fp8-support-on-h100/190236/7 "2024-03-08T17:18:20Z")
Is the mixed precision support planned for FP8? There was an announcement in JAN in the dev forum: > **[Float8 in PyTorch \[1/x\]](https://dev-discuss.pytorch.org/t/float8-in-pytorch-1-x/1815)** > > This note introduces the experimental support for float8 training in PyTorch. The float8 data formats (as laid out in https://arxiv.org/pdf/2209.05433.pdf) have been added in recent hardware such as NVIDIA Hopper. They have theoretical advantages... ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [March 8, 2024, 7:55pm UTC](https://discuss.pytorch.org/t/fp8-support-on-h100/190236/8 "2024-03-08T19:55:28Z")
[`TransformerEngine`](https://github.com/NVIDIA/TransformerEngine) supports FP8 for a long time now. Alternatively you can also play around with [`pytorch-labs/float8_experimental`](https://github.com/pytorch-labs/float8_experimental). ---
**Author:** ![bhack](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bhack/32/62677_2.png) [@bhack](https://discuss.pytorch.org/u/bhack)\ **Post date:** [March 8, 2024, 8:15pm UTC](https://discuss.pytorch.org/t/fp8-support-on-h100/190236/9 "2024-03-08T20:15:10Z")
Thanks, my point was more related to pytorch AMP support.