Hi everyone,
I’ve been experimenting with connecting my Mac mini M4 and a Linux machine with an NVIDIA GPU directly over Thunderbolt 4, with the goal of using both machines for distributed training and inference.
That led me to build TBCCL, a standalone C++ communication library that runs over TCP using Thunderbolt IP networking. I’ve also built torch-tbccl, which registers TBCCL as a backend for torch.distributed.
I’ve tested two-host communication and all-reduce on my Mac/Linux setup. There’s a setup guide for configuring the Thunderbolt connection, and the PyTorch adapter has an example that runs across both machines.
I’ve also been working on integrations for vLLM and Exo. The idea is to keep the communication runtime independent of individual frameworks so that it can be reused for training and inference.
It’s still experimental, and I haven’t done systematic performance comparisons against Gloo or other backends yet. That’s next on my list.
I’d be interested in feedback from anyone who’s worked on custom torch.distributed backends, particularly around heterogeneous device support and integration with PyTorch’s distributed APIs.