Warning and Error converting CenterNet model into ONNX

1

I’m trying to convert a CenterNet model into ONNX and then from ONNX to Caffe by using ONNXToCaffe, since the DCNv2 used in centernet is not yet available in Caffe so I only work on the version that doesn’t use it. More specifically, dlav0, in centernet.

I tried it in three ways:

  1. create and load model using the code provided in CenterNet src, and convert it into ONNX directly using torch.onnx.export; all works well, I got a ‘.onnx’ file
  2. create and load model using the code provided in CenterNet src, then trace the tensor operation using torch.jit.trace, save into ‘.pth’ file, reload the ‘.pth’ file; before tracing the tensor I rewrote the forward function because the original output is a dict, I need a tensor output to trace:
model = get_dlav0(34, {'hm': 80, 'wh': 2, 'reg': 2},head_conv=256)
def forward(self, x):
    x = self.base(x)
    x = self.dla_up(x[self.first_level:])
    ret = []
    for head in self.heads:
        ret.append(self.__getattr__(head)(x))
    return torch.cat(ret,1)
model.forward = MethodType(forward,model)
model = load_model(model,modelpath)
dummy_input = torch.rand(1, 3, 512, 512)
model.eval()
target = model(dummy_input)
model = torch.jit.trace(model,dummy_input)
torch.jit.save(model,jitpath)

Then I reload the jit.pth file to export ONNX, then I got the following warrning:

UserWarning: Model has no forward function
warnings.warn(“Model has no forward function”)

I still got the onnx file and tested it with some input, looks right. Wondering why does this warning occur

  1. I then tried similar stuff but with torch.jit.script this time, I got folloing RuntimeError:

RuntimeError:
Expected a default value of type Tensor (inferred) on parameter “residual”.Because “residual” was not annotated with an explicit type it is assumed to be type ‘Tensor’.:
File “D:\Code\Common-Model-Transformation-Service\src\dl2caffe\pytorch2caffe\CenterNet\src\lib\models\networks\dlav0.py”, line 42
def forward(self, x, residual=None):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
if residual is None:
~~~~~~~~~~~~~~~~~~~~
residual = x
~~~~~~~~~~~~

    out = self.conv1(x)
    ~~~~~~~~~~~~~~~~~~~
    out = self.bn1(out)
    ~~~~~~~~~~~~~~~~~~~
    out = self.relu(out)
    ~~~~~~~~~~~~~~~~~~~~

    out = self.conv2(out)
    ~~~~~~~~~~~~~~~~~~~~~
    out = self.bn2(out)
    ~~~~~~~~~~~~~~~~~~~

    out += residual
    ~~~~~~~~~~~~~~~
    out = self.relu(out)
    ~~~~~~~~~~~~~~~~~~~~

    return out
    ~~~~~~~~~~ <--- HERE

It has something to do with the residual block for DLA structure, any idea how to rewrite it?

I’m trying ‘trace’ or ‘script’ because I want someone else can use it without creating the model instance.

Doing everything on CPU, envs

ubuntu 18.04
pytorch 1.10.0

Thanks

2

Could you please show the code you used in the first way?