I am trying to do a bunch of algebra on data arrays being loaded from disk. So I have 2 steps: data_load and data_process. It turns out that the data_load takes about 50% of data_process (nice problem to have btw!). But this raises the possibility of loading the next batch to GPU while waiting for current batch to finish.
So… would like to figure out a way to run data_load(next_batch) (using multiprocessing?) plus transfer to GPU while data_process(current_batch) is running. I’m a bit confused whether this is even possible, and whether nn.DataParallel modules are meant for for this kind of single GPUwork.
I tried coding something simple using cpu multiprocessing, but I think it defeats the purpose as starting a pool is slow (I would need to do this moultiple times) - and I’m not even sure if this asynch call does what I’m hoping it will do.
pool = mp.Pool(processes = 1) data_next = pool.apply_async(load_data_next, args = (chunk_ctr, )) pool.close() pool.join() data[current_batch] = torch.from_numpy(data_next).float().to(device)
Any advice on this issue?
Thanks in advance!