Preprocess images on GPU

vision
1

In my preprocessing pipeline I am doing something like this:

preprocess = transforms.Compose([
            transforms.Scale(100),
            transforms.CenterCrop(100),
            transforms.ToTensor()
        ])

img = Image.open('image_path')
img_t = preprocess(img).cuda()

As far as I understand, the complete preprocessing would be performed by the CPU, as img is not a cuda Tensor. It will be moved to the GPU only after preprocessing. Is there any possibility to do this on GPU and only have the CPU do the read from disk?

7 Likes
2

In general, I wouldn’t recommend doing this. But I see no reason why you couldn’t if you defined your own transforms that only used tensor ops (so ToTensor would be first). The default ones use PIL. I’m not sure if this will cause any issues with the default batch collate though, it may expect data to be on the CPU. I haven’t looked closely

Either way, I would say don’t bother for this reason, GPU memory is a much more limited resource and CPU to GPU copy isn’t free. Better to use the default data pipeline and load, process, collate on the CPU and then copy to GPU. Especially if your input images are larger than your network input which is often the case. You don’t want 640x480 images being loaded on the the GPU if you’re cropping to 224x224 anyways.

The only ‘preprocessing’ type op I sometimes put on the GPU (as pseudo model layer) is per-image normalization (x - mean(x))/std(x) that operates on batches of images.

2 Likes