Hi everyone,
I am working on a high-performance multi-agent concept called megabots2026. The goal is to maximize prediction accuracy (targeting 100% verification on specific logical/deterministic tasks) by running a two-agent cooperative loop: Agent A (Generator) and Agent B (Verifier).
To achieve a x10 performance leap and eliminate Python’s runtime/async overhead, I want to implement the entire execution loop, state management, and agent communication layer directly in C++ using LibTorch.
The core loop works like this:
-
Agent A predicts an answer.
-
Agent B instantly validates it via shared memory.
-
If validation fails, Agent B feeds back parameters to Agent A to recalculate, looping at bare-metal speed until the confidence/accuracy threshold is met.
My questions for the LibTorch experts here:
-
To keep latency near zero, should I handle the tensor sharing between the two models using standard C++ threads (
std::thread) with sharedat::Tensorreferences, or is there a cleaner way to handle concurrent GPU streams within LibTorch? -
Has anyone benchmarked the raw overhead reduction when moving a tight multi-agent feedback loop completely out of Python and into native C++?
-
Looking for Collaborators: I am actively looking to connect with strong, serious AI and C++ developers who have experience with GPU-device optimization and high-performance computing to help build megabots2026 and make it incredibly powerful. If you want to push this architecture to its absolute limit, drop a comment below or send me a DM so we can team up!
Would love to hear thoughts on the memory architecture for this kind of setup!