Which CPU are the batch-fractions collated on?

Is it CPU 0? Does the data from the other CPUs get transferred onto it?