I followed the latest Nvidia conference and they announced an official PyTorch and TensorRT integration that should allow to convert PyTorch model with a single line of code. When will this be available? And what PyTorch version will include this feature?
Thank you
Here is the repository : GitHub - NVIDIA/Torch-TensorRT: PyTorch/TorchScript compiler for NVIDIA GPUs using TensorRT
Release notes : Release Torch-TensorRT v1.0.0 · NVIDIA/Torch-TensorRT · GitHub
@Dheeraj_Peri Linked the repo but to answer the questions:
When will this be available?
- The integration is distributed as a separate python package similar to
torchvision
ortorch-xla
. You can download Torch-TensorRT now. The following command will get you the latest release (the version mentioned at GTC)
pip3 install torch-tensorrt -f https://github.com/NVIDIA/Torch-TensorRT/releases
And what PyTorch version will include this feature?
- The prebuilt wheels target PyTorch version 1.10.0+cu113 (The cuda version is important for TensorRT and builds of Torch-TensorRT are PyTorch version specific).
- You can build the project from source for the specific version of PyTorch, CUDA, TensorRT and cuDNN you’d like fairly easily by modifying the WORKSPACE file (https://github.com/NVIDIA/Torch-TensorRT/blob/master/WORKSPACE)
- To support some older PyTorch versions might require small code changes, post a Github issue and we are happy to help you.
- There are older releases that target PyTorch versions back to PyTorch 1.4.0 if you quickly want to try out Torch-TensorRT but we would recommend you try to backport Torch-TensorRT v1.0.0 to an older PyTorch release because of the amount of features that have been added each version.
More information can be found here: https://nvidia.github.io/Torch-TensorRT/index.html
Also feel free to tag us if you have other questions.
1 Like