Inference my model on TRT Engine file

deployment
1

Hello,

I had converted my *.pt file to onnx and validated its inference. Everything was OK.
When I converted my Onnx file to TRT Engine file and did inference it on Jetson Nano.
The predicted output was ZERO matrix like the below (Even my image input has values).

Predicted output:
[[ 0. 0. 0. …, 0. 0. 0.]
[ 0. 0. 0. …, 0. 0. 0.]
[ 0. 0. 0. …, 0. 0. 0.]
…,
[ 0. 0. 0. …, 0. 0. 0.]
[ 0. 0. 0. …, 0. 0. 0.]
[ 0. 0. 0. …, 0. 0. 0.]]

Image input (as Numpy): For example
[[[ 0.73339844 0.74121094 0.74121094 …, 0.1451416 0.11767578
0.12158203]
[ 0.73339844 0.73339844 0.73339844 …, 0.08233643 0.06274414
0.10198975]
[ 0.73730469 0.72558594 0.74121094 …, 0.07452393 0.05883789
0.10198975] ]]

This is my inference function:
def predict(img):
cuda.memcpy_htod_async(d_input, img, stream)
context.execute_async_v2(bindings, stream.handle, None)
cuda.memcpy_dtoh_async(output, d_output, stream)
stream.synchronize()
return output

Please advise why the inference result was ZERO matrix. Thanks.

2

I would recommend to post the question in a TensorRT board or repository, as it doesn’t seem to be PyTorch-specific.

1 Like