Not using multiple GPUs

1

I set CUDA_VISIBLE_DEVICES=0,1,2. However, my code only uses GPU 0, and when it’s out of memory the program crashes. What’s the problem here?

2

Did you use:

model = torch.nn.DataParallel(model, device_ids=list(range(3)))
3

No I didn’t set model to be data parallel.

4

DataParallel would run the model on three batches in parallel, that won’t help with out of memory errors.

It would be possible to run different parts of a single model on several GPUs using part1.cuda(0), part2.cuda(1) etc. but then you would have to edit the forward method in order to shift the output of part1 to cuda(1) in order to input it to part2.

That would slow the model down considerably, and I am not certain that it would help with out of memory errors.