Can we have communication primitives such as send and recv when using the TensorPipe backend? I want to use it for inference between processes, having TensorPipe figure out the best way of communications. Is there any other way to implement it using the current RPC - TensorPipe api?
Hey @ItamarWilf,
Can we have communication primitives such as
sendandrecvwhen using the TensorPipe backend?
Yep, it should be possible to use ProcessGroup send/recv in conjunction with RPC. But we don’t yet have a TensorPipe backend for ProcessGroups. So the send/recv API needs to choose from gloo, nccl, and mpi backends.
One caveat is that, ProcessGroup send/recv requires the orders to match on the sender and receiver. However, RPC by-itself does not guarantee orders, as it will grab threads from a pool to process requests concurrently. So, if you are using ProcessGroup send/recv within RPC functions, you might need to enforce the order in user functions.
cc @lcw
I want to use it for inference between processes, having TensorPipe figure out the best way of communications. Is there any other way to implement it using the current RPC - TensorPipe api?
Yep, this should be possible. You can, e.g., using the RPC to implement a queue, and then call enqueue on the sender and dequeue on the receiver. Then enqueue/dequeue will be similar to send/receive.
Please let us know if this would work for you.
Echoing @mrshenli, if all you need is send and recv, you may be better off using the RPC primitives, which were designed for this type of point-to-point communication, rather than using the ProcessGroup interface. There are currently no plans to expose TensorPipe as a ProcessGroup backend.
1 Like