Bfloat16 type, cuda, pytorch or gpu issue

1

Hello. When I try to run following code snippet:

self.metric3d = torch.hub.load('yvanyin/metric3d', 'metric3d_vit_small', pretrain=True).to(self.device)
pred_depth, confidence, output_dict = self.metric3d.inference({'input': rgb})

I get following error:

raise RuntimeError('Current CUDA Device does not support bfloat16. Please switch dtype to float16.')
RuntimeError: Current CUDA Device does not support bfloat16. Please switch dtype to float16.

RGB image dtype is torch.float32. Is this error related to dtype of image, or that simply my graphics card does not support type bfloat16?

Please switch dtype to float16 => does this mean to switch dtype or RGB to torch.float16 in my code or it refers to code of your project?

If I cast my RGB to torch.float16 then error is:

RuntimeError: Input type (c10::Half) and bias type (float) should be the same

My graphics card is: GeForce RTX 2080 SUPER

So I would like to know how to solve issue and whether this issue is related to the:

  1. pytorch and specific version
  2. cuda and specific version
  3. gpu device itself (i.e. that gpu device simply does not support this type)
  4. something else
2

The error is raised from autocast using bfloat16 as it’s lower dtype, which is not supported on your GPU as bfloat16 operations were added for Ampere+ GPUs.
Thus is_bf16_supported fails and raises the error from here (assuming you are using a recent PyTorch release).

Based on the code you’ve provided it seems an autocast context is used inside the model itself and you would need to narrow down where it’s used and switch to float16.