It seems that model._distributed_broadcast_coalesced
will only broadcast rank-0’s weights to others, so how can I manage to broadcast another rank’s weights to other models?
Thanks
It seems that model._distributed_broadcast_coalesced
will only broadcast rank-0’s weights to others, so how can I manage to broadcast another rank’s weights to other models?
Thanks