Return list of future where callback or result set inside async block

distributed
1

Hi Folks,

Can I do return list of future to a master node meanwhile on workers perform async ? On a worker, I have foo and master issue rpc_async by some reason it doesn’t

ob_rref.rpc_async().foo(self.agent_rref, n_steps, meta_tasks)

@rpc.functions.async_execution
def foo(self, agent_rref, n_steps, task_id):

       futures = []
        for i in range(self.num_task):
            print(f"Create feature {i}")
            fut = torch.futures.Future()
            futures.append(fut)

        self.loop.run_until_complete(self.bar(futures))
        return torch.futures

Bar run in own loop, since if I undestood mp.span create either fork or thread but in all cases it separate thread so loop on each worker are different.

async def bar(self, futures):
      # here I set each rpc future that I returned.
      futures[i].set_result(Something)

      # i.e I have

     while True:
        async_future = await queue.get()
        if async_future is None:
            break

       futures[i].set_result(async_future)