Does DistributedDataParallel split data at the batch dimension?

Thanks for the clarification. Just to confirm my understanding: that means each node will work with a separate data loader and thus each node the batch size will be 64, am I right?