How to track node activity in distributed training with torch.distributed

When using torch.distributed over the network (distributing training over machines not on the same lan) is there a way that I can measure what the contribution of each machine is? I’d like to measure flops or some proxy for flops to determine which machines bear the most load.