Yes, usually you would load the data on the CPU and push it to the GPU inside the training loop in order to overlap the data loading with the actual model training.
However, if you want to load CUDATensors directly, check the CUDA multiprocessing docs to avoid re-creating a CUDA context, which raises your error.
1 Like