TensorRT integration release date

I followed the latest Nvidia conference and they announced an official PyTorch and TensorRT integration that should allow to convert PyTorch model with a single line of code. When will this be available? And what PyTorch version will include this feature?
Thank you

Here is the repository : GitHub - NVIDIA/Torch-TensorRT: PyTorch/TorchScript compiler for NVIDIA GPUs using TensorRT
Release notes : Release Torch-TensorRT v1.0.0 · NVIDIA/Torch-TensorRT · GitHub

@Dheeraj_Peri Linked the repo but to answer the questions:

When will this be available?

  • The integration is distributed as a separate python package similar to torchvision or torch-xla. You can download Torch-TensorRT now. The following command will get you the latest release (the version mentioned at GTC)
pip3 install torch-tensorrt -f https://github.com/NVIDIA/Torch-TensorRT/releases

And what PyTorch version will include this feature?

  • The prebuilt wheels target PyTorch version 1.10.0+cu113 (The cuda version is important for TensorRT and builds of Torch-TensorRT are PyTorch version specific).
  • You can build the project from source for the specific version of PyTorch, CUDA, TensorRT and cuDNN you’d like fairly easily by modifying the WORKSPACE file (https://github.com/NVIDIA/Torch-TensorRT/blob/master/WORKSPACE)
    • To support some older PyTorch versions might require small code changes, post a Github issue and we are happy to help you.
  • There are older releases that target PyTorch versions back to PyTorch 1.4.0 if you quickly want to try out Torch-TensorRT but we would recommend you try to backport Torch-TensorRT v1.0.0 to an older PyTorch release because of the amount of features that have been added each version.

More information can be found here: https://nvidia.github.io/Torch-TensorRT/index.html
Also feel free to tag us if you have other questions.

1 Like