Can't convert cuda:0 device type tensor to numpy When trying to enumerate through torch.utils.data.DataLoader object

[weirenorweiren]

Hi @ptrblck, thanks so much for your detailed explanations and they are super helpful!

Regarding the 3rd point, I tried to move the data to GPU for dataset but there is a bug that RuntimeError: Caught RuntimeError in DataLoader worker process 0.. Then I found that you answered a same problem one year ago! I quoted it below to supplement your answer and hope it could benefit others with similar type of questions.

@ptrblck

Yes, usually you would load the data on the CPU and push it to the GPU inside the training loop in order to overlap the data loading with the actual model training.
However, if you want to load CUDATensors directly, check the CUDA multiprocessing docs to avoid re-creating a CUDA context, which raises your error.

Thanks again for your prompt and patient response!