Converting model that has dynamic control flow to load in C++

torch.compile
1

Hello,

I’m working on converting a PyTorch model to a format that can be loaded in C++. My model contains dynamic control flow (for loops and if blocks), so I’ve been using TorchDynamo and TorchInductor for optimization. However, I’m encountering difficulties in finding a way to load these optimized models in C++.

I’ve tried the following approaches:

  1. Using TorchDynamo and TorchInductor to optimize the model
  2. Attempting to use TorchScript on the optimized model
  3. Exploring AOT Inductor to convert the model to .pt format

Unfortunately, none of these methods have been successful in producing a .pt file that can be loaded in C++.

My questions are:

  1. Is it currently possible to load TorchDynamo/TorchInductor-optimized models with dynamic control flow in C++?
  2. If so, what is the recommended approach or workflow to achieve this?