Any general guidelines both space and time wise as to which frameworks are best suited in the various stages of ML
Are there any guidelines as to why transforms, for example, shouldn’t happen on the GPU but on Pytorch CPU. If Pytorch CPU is slower than Numpy, then why not code the transforms up in Numpy.
I would claim it all depends on your use case and there is no silver bullet.
By default you would want to use the GPU to train the model and the CPU to load and process the data. Using multiple workers in the DataLoader the data loading and processing will be applied in the background, while the GPU is busy with training the model.
torchvision.transforms are using PIL as the default backend for image transformations, which uses numpy arrays internally.
However, if you need differentiable transformations (e.g. via Kornia), you would have to apply them on tensors and cannot use numpy anymore (or would need to write the backward functions manually).
Also, on “bigger” systems, you might see that the CPU is not fast enough feeding the GPUs, which is visible in starving GPUs.
In that case you could use e.g. NVIDIA DALI to apply the transformations on the GPU, which could give you an overall performance boost.
That’s what I needed. Thanks!