Float32 to Bfloat16 conversion

Arul · April 20, 2023, 7:44pm

When pytorch converts fp32 to bfloat16, does it do truncation or rounding by default?

x = torch.randn([3,4]) # fp32
x.to(torch.bfloat16) #bfloat16

pytorch/pytorch/blob/main/c10/util/BFloat16.h#L45


      
            // but it fails in the HIP environment.
            tempRes = reinterpret_cast<float*>(&tmp);
            res = *tempRes;
          #else
            std::memcpy(&res, &tmp, sizeof(tmp));
          #endif
          
          
  return res;
          }
          
          
inline C10_HOST_DEVICE uint16_t bits_from_f32(float src) {
            uint32_t res = 0;
          
          
#if defined(USE_ROCM)
            // We should be using memcpy in order to respect the strict aliasing rule
            // but it fails in the HIP environment.
            uint32_t* tempRes = reinterpret_cast<uint32_t*>(&src);
            res = *tempRes;
          #else
            std::memcpy(&res, &src, sizeof(res));
          #endif

I see that it has utility functions to do both but how can I find which gets triggered by default?

eqy · April 20, 2023, 9:56pm

I think you can write a simple script to test this:

import torch

elemns = 10
a = torch.rand(elemns, dtype=torch.float32)
b = a.bfloat16()
c = b.float()
print(f"got {(c>a).sum()} elements rounded up out of {elemns}")
print(c, a)

got 6 elements rounded up out of 10
tensor([0.0253, 0.2080, 0.1826, 0.1118, 0.3809, 0.4434, 0.5742, 0.9453, 0.8789,
        0.4004]) tensor([0.0253, 0.2080, 0.1821, 0.1119, 0.3806, 0.4438, 0.5732, 0.9467, 0.8782,
        0.3997])