Traffic of Distributed Data Parallel (DDP) using CPU

while using Distributed data parallel of Pytorch on CPU, I used CIFAR10 dataset , I run on two servers just with cpu and captured the traffic . i have two question, first is the master node during the whole process also does the computation and meanwhile sending packets to worker or no? second is why there is heavy flow always and why it doesn’t have steady line then sometimes during each epoch it doesn’t make peaks and it has always heavy flow? because I was expecting a peak on each epoch that nodes do the backward pass but there are always packets going and returning

*I know the CPU is not suitable for this reason but it’s just an experiment