How to gather not tensor object in torch.distributed?

1

What I want to do is gather not tensor object in torch.distributed. For example, I launch job using torchrun and have these dicts on 4 ranks:

P 0: {0: 1}
P 2: {2: 1}
P 4: {4: 1}
P 6: {6: 1}
P 8: {8: 1}
All: {}

what I want is

P 0: {0: 1}
P 2: {2: 1}
P 4: {4: 1}
P 6: {6: 1}
P 8: {8: 1}
All: {0: 1, 2: 1, 4: 1, 6: 1, 8: 1}

Is there any way to do this?