TensorRT: Errors in PTQ Example

I was trying to follow the TensorRT PTQ examples here to implement INT8 quantization using TensorRT:
https://pytorch.org/TensorRT/ts/ptq.html#how-to-create-your-own-ptq-application-in-python

However, if I try to use torch_tensorrt.ptq.DataLoaderCalibrator() and torch_tensorrt.ptq.CalibrationAlgo.ENTROPY_CALIBRATION_2, I get following error message: AttributeError: module ‘torch_tensorrt’ has no attribute ‘ptq’.

I found in another place that the ptq module was available under ts (seems to be up-to-date w.r.t. the Github repo), so I also tried:
torch_tensorrt.ts.ptq.DataLoaderCalibrator()
and torch_tensorrt.ts.CalibrationAlgo.ENTROPY_CALIBRATION_2,
but the result was the same:
AttributeError: module ‘torch_tensorrt.ts’ has no attribute ‘ptq’

I am using following versions under Python 3.11:

  • tensorrt 10.3.0
  • torch 2.5.1
  • torch_tensorrt 2.5.0
  • nvidia-modelopt 0.21.1

I think the examples given under Post Training Quantization (PTQ) — Torch-TensorRT v2.6.0.dev0+69c83d4 documentation don’t refer to the current module structure.
Can you give me a hint here and possibly update the example?

Cheers,
Tim

CC @narendasan for TorchTRT

The quantization API from TensorRT has been deprecated so this API doesn’t really work anymore and will be removed soon. You can use model-opt + the dynamo frontend to do this now. Deploy Quantized Models using Torch-TensorRT — Torch-TensorRT v2.6.0.dev0+69c83d4 documentation

Hi Naren, thanks a lot - the using model-opt & dynamo frontend works for me!