How to apply model to multi-device with pytorch1.0?

1

I am reading the documentation of nn.DataParallel:

  • device_ids ( list of python:int or torch.device ) – CUDA devices (default: all devices)

And I can only set single device with code like device = torch.device("cuda:0").

How to set multi-device to use the default: all devices?

2

If you set device = None it uses all (None is the option by default in fact)

3

If I want to only use 4 gpus in a server with 8 gpus, how should I do that?

4

it’s better you to use the environment variable
CUDA_VISIBLE_DEVICES = id1,id2 and so on
it’s a cuda flag which will make any program to detect only those gpus you choose.

5 
CUDA_VISIBLE_DEVICES=0,2,3,7 python your_script.py
6

The input tensor should also be put on gpu devices, I can do this with input_tensor.to(device) if there is only single gpu device. How should I do that with multi-gpu?

7

The input tensor should also be put on gpu devices, I can do this with input_tensor.to(device) if there is only single gpu device. How should I do that with multi-gpu?

8

If you are using dataparallel you don’t really need to move the tensor into devices, it’s done automatically once you feed the forward function.