Realtime capabilities of TorchScript / memory allocations

At the moment I am exploring the possibility of writing controllers for a real robot hardware using TorchScript. One of the requirements is to have low latency and constant execution times for the script. That is as the robot controller is running in a realtime environment with hard 1 kHz (=1 ms) time limits.

Does someone of you know how the JIT compiler of TorchScript handles memory allocations? Is there something like a “realtime” mode switch to avoid memory allocations or does this all depends on the implementation of the TorchScript C++ operators?


The JIT has no realtime mode. Allocation is controlled by operator implementations, which are generally (for CPU ops) tuned for server workloads.

1 Like