PyTorch ONNX export issue: torch.cat( (y1, y2, y2), 1 ) is being exported as onnx::Constant[value=<Tensor>]

1

I am trying to export my own implemented YOLOv3 from PyTorch to ONNX (To be used in Caffe2+TensorRT). The outputs of the network are like this:

y1 = self.yolo_82(x)
...
y2 = self.yolo_94(x)
...
y3 = self.yolo_106(x)
return torch.cat( (y1,y2,y3), 1 )

The issue is, the ONNX exporter is exporting this last layer as:

{
  %439 : Float(1, 10647, 85) = onnx::Constant[value=<Tensor>]()
  return (%439);
}

As a result when I run the model in Caffe2, it gives me the same output as on the image I used to export the network.
My questions are:

  1. PyTorch Docs claim that torch.cat is supported in ONNX export, if yes then why is the ONNX export returning a constant array instead of concatenated tensors?
  2. Is there some special/secret ingredient that I am missing here.

Any/all help is appreciated as I am stuck here for quite long.

2

@khurram.amin could you provide the full code, so I can reproduce it. Thanks!

3

There was a bug in my code, I was using .detach() and .data in my forward pass. Removing it fixed the constant issue.

4

How did you use detach and .data? Could I see your code?

5

@Rizhao_Cai I don’t remember the exact code/bug in my code since its been a few months old. But basically somewhere in the code I was detaching the tensor from the graph that PyTorch uses to compute auto-grads.
Example:

tensor_I_want_in_my_graph = torch.FloatTensor([1])
modified_tensor = tensor_I_want_in_my_graph.detach()
...

As a result of .detach() pyTorch removes that tensor from backprop calculation and the pyTorch to ONNX export treats such a tensor as a constant tensor and just remembers the values instead of remembering the mathematical operation.
Documentation of detach() method

The .data operation is a bit different. Since each tensor is an object of class Tensor(), it has a property .data which contains the values of the tensor. I am not sure about the exporting behavior of .data in pyTorch + ONNX. You can do a small experiment to test it out.