Hey,
I’ve been working with C++ and LibTorch for a while, and kept running into the same recurring issues — figuring out the correct build for each OS, switching between CUDA, MPS, and CPU backends, and dealing with GPU-related crashes that could take down the entire application. We decided to build a project to simplify this process and make deployment more reliable.
It’s called blace.ai. Think of it as a wrapper around LibTorch + ONNX Runtime with:
-
Same API across Windows, Linux, macOS (Intel + Apple Silicon)
-
TorchScript CPU/CUDA/MPS, ONNX CPU, DirectML all supported without touching the model code
-
Process isolation and serialization of AI workloads, so crashes/leaks don’t kill your main app
-
No Python anywhere, all deps bundled
-
Computation graphs: Chain multiple models/ops together (e.g., run a detector → segmenter → postprocess) with auto tensor caching so you don’t waste time re-running parts of the pipeline
-
A small but growing “model hub” with pre-optimized TorchScript/ONNX models you can drop in — we’re actively expanding the hub and optimizing models for real-world deployment
Here’s a quickstart example if you’re curious.