I want to transfer my data preprocession into GPU, and I have writen a series of code to do something to a torch.cuda.tensor before feeding it into network, the code was proved to be true. But when I actually use this way to train my model, the cuda runtime initialization error always occure. I can’t understand where the problem is.
The error massage is
RuntimeError: cuda runtime error (3) : initialization error at /home/ayu/pytorch/aten/src/THC/THCCachingAllocator.cpp:508
Before I fed data to model, the defined model has been transfered to GPU, I don’t know if this is the problem.
Thank you for your advice!
Are you using multiple workers in your DataLoader?
Could you share some code of your Dataset and the instantiation of your DataLoader?
Your error might be related to some multi-processing / CUDA issue.
Yes I used multiple workers to get data for network. Now I choose to get data from DataLoader and then do some transform for a batch of data to bypass this problem.
Thank you for your reply.