Hi,
I need to compile a model for a range of input sizes with a step size. For example, the model should be compiled for the following 4 sizes where step size is 12.
1x256x192x12
1x256x192x24
1x256x192x36
1x256x192x48
If I try to compile with min and max shapes (1x256x192x12 and 1x256x192x48), it doesn’t work because the model fails to make the inference for some sizes due to the torch.cat operation. Is there a way to resolve this issue without changing the model?
Also, when compiling a model with torch_tensorrt like the following, does the value of torch.backends.cudnn.benchmark matter?
torch_tensorrt.compile(self.backbone_2d, ir=‘dynamo’, inputs=inp)
Making it False causes the following warning:
WARNING:py.warnings:/home/humble/.local/lib/python3.10/site-packages/torch/nn/modules/conv.py:456: UserWarning:
Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at /opt/pytorch/aten/src/ATen/nat
ive/cudnn/Conv_v8.cpp:919.)
return F.conv2d(input, weight, bias, self.stride,