ONNX export complex network

Hi,

I tried to follow this tutorial (https://pytorch.org/tutorials/advanced/super_resolution_with_caffe2.html) on converting a pytorch model to onnx for mobile usage. I used the current nightly build (shipped via pip). It worked fine with a simple model like

class RegNet(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.features = torch.nn.Sequential(
            torch.nn.Conv2d(3, 6, 5),
            torch.nn.ReLU(),
            torch.nn.MaxPool2d(2, 2),
            torch.nn.Conv2d(6, 16, 5),
            torch.nn.ReLU(),
            torch.nn.MaxPool2d(2, 2)
        )
        self.classifier = torch.nn.Sequential(
            torch.nn.Linear(16 * 5 * 5, 120),
            torch.nn.BatchNorm1d(120),
            torch.nn.ReLU(),
            torch.nn.Dropout(0.25),
            torch.nn.Linear(120, 84),
            torch.nn.BatchNorm1d(84),
            torch.nn.ReLU(),
            torch.nn.Linear(84, 1)
        )

    def forward(self, x):
        x = self.features(x)
        x = x.view(x.size(0), -1)
        x = self.classifier(x)
        return x

but did not work with a more complicated model like the U-Net-Generator from this repo (https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/blob/master/models/networks.py) (line 237).

My approach was both times the same:
torch.onnx.export(net, torch.rand(1, 3, 480, 480), "tmp.onxx", export_params=True) (didn’t work with torch.onnx._export(...) either.

The error message is quite long:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/schock/anaconda3/envs/pytorch_nightly/lib/python3.6/site-packages/torch/onnx/__init__.py", line 22, in _export
    return utils._export(*args, **kwargs)
  File "/home/schock/anaconda3/envs/pytorch_nightly/lib/python3.6/site-packages/torch/onnx/utils.py", line 286, in _export
    proto, export_map = graph.export(params, _onnx_opset_version, defer_weight_export, operator_export_type)
RuntimeError: block->inputs().size() >= num_initializers ASSERT FAILED at /pytorch/torch/csrc/jit/export.cpp:284, please report a bug to PyTorch. (EncodeBlock at /pytorch/torch/csrc/jit/export.cpp:284)
frame #0: <unknown function> + 0x3fb4b7 (0x7ff7ddf9c4b7 in /home/schock/anaconda3/envs/pytorch_nightly/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #1: torch::jit::ExportGraph(std::shared_ptr<torch::jit::Graph> const&, std::vector<at::Tensor, std::allocator<at::Tensor> > const&, long, bool, torch::onnx::OperatorExportTypes) + 0x37 (0x7ff7ddf9c617 in /home/schock/anaconda3/envs/pytorch_nightly/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #2: <unknown function> + 0x374ee8 (0x7ff7debeeee8 in /home/schock/anaconda3/envs/pytorch_nightly/lib/python3.6/site-packages/torch/_C.cpython-36m-x86_64-linux-gnu.so)
frame #3: <unknown function> + 0xe12fd (0x7ff7de95b2fd in /home/schock/anaconda3/envs/pytorch_nightly/lib/python3.6/site-packages/torch/_C.cpython-36m-x86_64-linux-gnu.so)
frame #4: _PyCFunction_FastCallDict + 0x154 (0x55b6a21859e4 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #5: <unknown function> + 0x19cf4e (0x55b6a2212f4e in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #6: _PyEval_EvalFrameDefault + 0x2fa (0x55b6a223794a in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #7: PyEval_EvalCodeEx + 0x966 (0x55b6a220e2f6 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #8: <unknown function> + 0x198bd6 (0x55b6a220ebd6 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #9: PyObject_Call + 0x3e (0x55b6a21857ee in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #10: _PyEval_EvalFrameDefault + 0x1abb (0x55b6a223910b in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #11: <unknown function> + 0x196206 (0x55b6a220c206 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #12: <unknown function> + 0x1971cf (0x55b6a220d1cf in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #13: <unknown function> + 0x19ced5 (0x55b6a2212ed5 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #14: _PyEval_EvalFrameDefault + 0x10c5 (0x55b6a2238715 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #15: PyEval_EvalCodeEx + 0x329 (0x55b6a220dcb9 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #16: PyEval_EvalCode + 0x1c (0x55b6a220ea4c in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #17: <unknown function> + 0x1bf37b (0x55b6a223537b in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #18: _PyCFunction_FastCallDict + 0x91 (0x55b6a2185921 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #19: <unknown function> + 0x19cdfc (0x55b6a2212dfc in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #20: _PyEval_EvalFrameDefault + 0x2fa (0x55b6a223794a in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #21: <unknown function> + 0x196f8b (0x55b6a220cf8b in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #22: <unknown function> + 0x19ced5 (0x55b6a2212ed5 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #23: _PyEval_EvalFrameDefault + 0x2fa (0x55b6a223794a in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #24: <unknown function> + 0x196206 (0x55b6a220c206 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #25: <unknown function> + 0x1971cf (0x55b6a220d1cf in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #26: <unknown function> + 0x19ced5 (0x55b6a2212ed5 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #27: _PyEval_EvalFrameDefault + 0x2fa (0x55b6a223794a in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #28: <unknown function> + 0x196f8b (0x55b6a220cf8b in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #29: <unknown function> + 0x19ced5 (0x55b6a2212ed5 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #30: _PyEval_EvalFrameDefault + 0x2fa (0x55b6a223794a in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #31: <unknown function> + 0x196f8b (0x55b6a220cf8b in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #32: <unknown function> + 0x19ced5 (0x55b6a2212ed5 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #33: _PyEval_EvalFrameDefault + 0x2fa (0x55b6a223794a in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #34: <unknown function> + 0x196206 (0x55b6a220c206 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #35: <unknown function> + 0x1971cf (0x55b6a220d1cf in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #36: <unknown function> + 0x19ced5 (0x55b6a2212ed5 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #37: _PyEval_EvalFrameDefault + 0x2fa (0x55b6a223794a in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #38: <unknown function> + 0x196f8b (0x55b6a220cf8b in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #39: <unknown function> + 0x19ced5 (0x55b6a2212ed5 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #40: _PyEval_EvalFrameDefault + 0x2fa (0x55b6a223794a in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #41: <unknown function> + 0x196206 (0x55b6a220c206 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #42: <unknown function> + 0x1971cf (0x55b6a220d1cf in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #43: <unknown function> + 0x19ced5 (0x55b6a2212ed5 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #44: _PyEval_EvalFrameDefault + 0x2fa (0x55b6a223794a in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #45: PyEval_EvalCodeEx + 0x329 (0x55b6a220dcb9 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #46: PyEval_EvalCode + 0x1c (0x55b6a220ea4c in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #47: <unknown function> + 0x214c44 (0x55b6a228ac44 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #48: PyRun_FileExFlags + 0xa1 (0x55b6a228b041 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #49: PyRun_SimpleFileExFlags + 0x1c4 (0x55b6a228b244 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #50: Py_Main + 0x624 (0x55b6a228ed24 in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #51: main + 0xee (0x55b6a215675e in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)
frame #52: __libc_start_main + 0xe7 (0x7ff7f3404b97 in /lib/x86_64-linux-gnu/libc.so.6)
frame #53: <unknown function> + 0x1c847b (0x55b6a223e47b in /home/schock/anaconda3/envs/pytorch_nightly/bin/python)

Is this a known bug (and is there a workaround) or should I file an issue?

@justusschock thanks for reporting this. Could you provide the code snippet to reproduce the error?

Thanks,
Lu

Hi Lu,

Sorry that it took so long.

The script is given in the linked gist

Yeah, but I am wondering what configuration I can use to reproduce the problem. Such as input size, etc. Otherwise, I may get different error message.

This is given in the if __name__ == '__main__' closure.

You just have to run the script with the current pytorch 1.0 nightly (installed on Ubuntu 18.04 via pip without CUDA) to reproduce the error.

@justusschock: Have you solve the problem? I faced same issue

Were you able to export the onnx to caffe2?

I haven’t been able to solve the issue. My workaround is to use libtorch.

@justusschock: Can libtorch convert to caffe2 from pytorch weight? My aim is that use caffe2 to load pytorch trained model. Thanks

I don’t know whether this is possible. I used libtorch for deployment directly.

Is it faster than pytorch inference? My inference code using pytorch is too slow

This is probably due to your model definition and not related to pytorch itself. In general libtorch is only a C++ API to the backend which is also used by the Python API. I found it to be as fast as caffe (which makes sense, since the backends have been/are on process to/will be merged)

1 Like

Good point. How to check “due to your model definition and not related to pytorch itself.”?

You could post your model definition and we could have a look at it together or you could try the pytorch profiler