TensorRT integration release date

I followed the latest Nvidia conference and they announced an official PyTorch and TensorRT integration that should allow to convert PyTorch model with a single line of code. When will this be available? And what PyTorch version will include this feature?
Thank you

Here is the repository : GitHub - NVIDIA/Torch-TensorRT: PyTorch/TorchScript compiler for NVIDIA GPUs using TensorRT
Release notes : Release Torch-TensorRT v1.0.0 · NVIDIA/Torch-TensorRT · GitHub

@Dheeraj_Peri Linked the repo but to answer the questions:

When will this be available?

  • The integration is distributed as a separate python package similar to torchvision or torch-xla. You can download Torch-TensorRT now. The following command will get you the latest release (the version mentioned at GTC)
pip3 install torch-tensorrt -f https://github.com/NVIDIA/Torch-TensorRT/releases

And what PyTorch version will include this feature?

  • The prebuilt wheels target PyTorch version 1.10.0+cu113 (The cuda version is important for TensorRT and builds of Torch-TensorRT are PyTorch version specific).
  • You can build the project from source for the specific version of PyTorch, CUDA, TensorRT and cuDNN you’d like fairly easily by modifying the WORKSPACE file (Torch-TensorRT/WORKSPACE at master · NVIDIA/Torch-TensorRT · GitHub)
    • To support some older PyTorch versions might require small code changes, post a Github issue and we are happy to help you.
  • There are older releases that target PyTorch versions back to PyTorch 1.4.0 if you quickly want to try out Torch-TensorRT but we would recommend you try to backport Torch-TensorRT v1.0.0 to an older PyTorch release because of the amount of features that have been added each version.

More information can be found here: Torch-TensorRT — Torch-TensorRT master documentation
Also feel free to tag us if you have other questions.

1 Like