I have a 1D torch.tensor
, which length is uncertain. How I could broadcast it?
Currently, I propose one way, is it suitable?
def run(rank, world_size):
if rank == 0:
n_id = uncertain length 1D tensor
else:
n_id = torch.zeros(maxlength of n_id, dtype=torch.int64).to(rank)
dist.broadcast(n_id, src=0)
n_id = n_id[n_id.nonzero()]