I plan to use multiple gpus to train a CNN. The input to the CNN has the shape (N, C, H, W), where N contains images from M videos, the images from each video are placed in order of increasing N. For my application, I need to make sure that before and after the CNN (which is a VGG), the order of the images do not change. So I am wondering if N gets split to multiple GPUs and passed through the CNN, after re-collect them at the output, will the order of the images change?
No, they don’t change.