I’m getting the following error:
AttributeError: module 'tensorrt' has no attribute 'bf16'
While executing %clone_1 : [num_users=1] = call_function[target=torch.ops.aten.clone.default](args = (%expand,), kwargs = {memory_format: torch.contiguous_format, _itensor_to_tensor_meta: {<tensorrt_bindings.tensorrt.ITensor object at 0x1544e7d99af0>: ((6, 37, 192, 336, 15), torch.bfloat16, False, (35804160, 64512, 336, 1, 2386944), None, False, {}), <tensorrt_bindings.tensorrt.ITensor object at 0x1544e7d996b0>: ((6, 37, 192, 336, 15), torch.bfloat16, False, (35804160, 64512, 336, 1, 2386944), None, False, {})}})
When using torch.compile
with the tensorrt backend:
import torch_tensorrt
my_model = torch.compile(my_model, mode="reduce-overhead", fullgraph=true, backend="tensorrt")