How to convert pt to onnx

hello, I use the https://github.com/quark0/darts/tree/master/rnn and simplify the train/test/valid data to get a rnn model, I could get the saved pt files then, but always failed to save it to onnx with error like:

ValueError: NestedIOFunction doesn’t know how to process an input object of type torch.LongTensor

What code are you using to save the model to ONNX? I couldn’t find a reference to ONNX in the git you shared.

model_pt_path = "test_1.onnx"
data_1 = torch.randn(23, 64)
hidden_1 = torch.randn(1, 64, 850)
output = torch.onnx.export(model, (data_1, hidden_1), model_onnx_path, verbose=True)

I added code above to try to export the model, thanks a lot!

1 Like

I’ve just read the project requirements, you probably use PyTorch version 0.3.1 as recommended? Then I think this answer might be relevant.

Try to wrap your tensors data_1 and hidden_1 in torch.Variable() calls, maybe that will work.

PyTorch versions prior to 0.4.0 used Variable to indicate that a Tensor can be “backwarded” upon, whereas normal Tensors cannot.

Alex, thanks a lot!
I modified the code as belew, although the original error gone, but will get a new error:

    data_1 = torch.randn(23, 64)
    hidden_1 = torch.randn(1, 64, 850)
    variable_data_1 = Variable(data_1)
    variable_hidden_1 = Variable(hidden_1)
    output = torch.onnx.export(model, (variable_data_1, variable_hidden_1), model_onnx_path, verbose=True)
Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydev_run_in_console.py", line 52, in run_file
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/train.py", line 145, in <module>
    output = torch.onnx.export(model, (variable_data_1, variable_hidden_1), model_onnx_path, verbose=True)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/onnx/__init__.py", line 75, in export
    _export(model, args, f, export_params, verbose, training)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/onnx/__init__.py", line 116, in _export
    trace, torch_out = torch.jit.trace(model, args)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 241, in trace
    return TracedModule(f, nderivs=nderivs)(*args, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 268, in forward
    trace, (out_vars, out_struct) = traced_inner(in_vars, in_struct)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 287, in wrapper
    out_vars, out_struct = f(in_vars, in_struct)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 262, in traced_inner
    return _flatten(self.inner(*args, **kwargs))
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 355, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 345, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/model.py", line 131, in forward
    emb = embedded_dropout(self.encoder, input, dropout=self.dropoute if self.training else 0)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/utils.py", line 69, in embedded_dropout
    embed.scale_grad_by_freq, embed.sparse
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/_functions/thnn/sparse.py", line 59, in forward
    output = torch.index_select(weight, 0, indices.view(-1))
TypeError: torch.index_select received an invalid combination of arguments - got (torch.FloatTensor, int, !torch.FloatTensor!), but expected (torch.FloatTensor source, int dim, torch.LongTensor index)

so modified to

    data_1 = torch.randn(23, 64).long()
    hidden_1 = torch.randn(1, 64, 850).long()
    variable_data_1 = Variable(data_1)
    variable_hidden_1 = Variable(hidden_1)
    output = torch.onnx.export(model, (variable_data_1, variable_hidden_1), model_onnx_path, verbose=True)

will get error:

Traceback (most recent call last):
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/train.py", line 288, in <module>
    train()
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/train.py", line 225, in train
    output = torch.onnx.export(model, (variable_data_1, variable_hidden_1), model_onnx_path, verbose=True)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/onnx/__init__.py", line 75, in export
    _export(model, args, f, export_params, verbose, training)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/onnx/__init__.py", line 116, in _export
    trace, torch_out = torch.jit.trace(model, args)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 241, in trace
    return TracedModule(f, nderivs=nderivs)(*args, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 268, in forward
    trace, (out_vars, out_struct) = traced_inner(in_vars, in_struct)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 287, in wrapper
    out_vars, out_struct = f(in_vars, in_struct)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/jit/__init__.py", line 262, in traced_inner
    return _flatten(self.inner(*args, **kwargs))
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 355, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/modules/module.py", line 345, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/model.py", line 131, in forward
    emb = embedded_dropout(self.encoder, input, dropout=self.dropoute if self.training else 0)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/utils.py", line 69, in embedded_dropout
    embed.scale_grad_by_freq, embed.sparse
  File "/Users/fjbnu/anaconda2/envs/darts/lib/python3.6/site-packages/torch/nn/_functions/thnn/sparse.py", line 59, in forward
    output = torch.index_select(weight, 0, indices.view(-1))
RuntimeError: index out of range at /Users/soumith/minicondabuild3/conda-bld/pytorch_1518385717421/work/torch/lib/TH/generic/THTensorMath.c:277

Also I found that seems PyTorch 0.3.1 does not support rnn model export to onnx very well as https://github.com/pytorch/pytorch/issues/5234

So I used the PyTorch 0.4.1/1.1.0 to have a try, but will get errors :

Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydev_run_in_console.py", line 52, in run_file
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/train.py", line 145, in <module>
    output = torch.onnx.export(model, (variable_data_1, variable_hidden_1), model_onnx_path, verbose=True)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/onnx/__init__.py", line 26, in export
    return utils.export(*args, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/onnx/utils.py", line 94, in export
    operator_export_type=operator_export_type)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/onnx/utils.py", line 226, in _export
    example_outputs, propagate)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/onnx/utils.py", line 177, in _model_to_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args, training)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/onnx/utils.py", line 144, in _trace_and_get_graph_from_model
    trace, torch_out = torch.jit.get_trace_graph(model, args)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/jit/__init__.py", line 77, in get_trace_graph
    return LegacyTracedModule(f)(*args, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/jit/__init__.py", line 109, in forward
    out = self.inner(*trace_inputs)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 475, in __call__
    result = self._slow_forward(*input, **kwargs)
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 465, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/model.py", line 131, in forward
    emb = embedded_dropout(self.encoder, input, dropout=self.dropoute if self.training else 0)
  File "/Users/fjbnu/Downloads/darts/rnn/search-EXP-20190515-233237/scripts/utils.py", line 67, in embedded_dropout
    X = embed._backend.Embedding.apply(words, masked_embed_weight,
  File "/Users/fjbnu/anaconda2/envs/darts_0.4/lib/python3.6/site-packages/torch/nn/backends/backend.py", line 10, in __getattr__
    raise NotImplementedError
NotImplementedError

The last error with PyTorch 0.4.1 / 1.1.0 is already clearer than the previous ones. Making progress!

This seems like some module in PyTorch isn’t supported by ONNX yet, maybe… The second last line (X = embed._backend...) in your stacktrace appears to call the backend’s Embedding module, and I have no idea what that could be and how to debug it.

I’m afraid that’s as far as I can help you :confused: Maybe someone else had a similar problem and can comment here?