DataParrallel Invalid Device Ids

distributed
1

Hi,

I have access to 4 GPUs, and would like to use 2 for my task training with DataParrallel. From what I understand, instructing torch which GPUs to use is done by:

task = nn.DataParrallel(model, device_ids=[1, 2]) to use GPUs 1 and 2.

However, this yields the following assertion error: “raise AssertionError(“Invalid device id”)”.

From my looking at previous answers (apologize I’m not sure how to link them, first post), this error indicates that I do not have GPUs 1 or 2 available to me. However, this does not appear to be the case.

For instance, extracting the device properties in the python interpreter (same environment) works without issue:

56%20PM
56%20PM1517×47 4.85 KB

Is there something I’m missing with regards to assigning the model to specific device ids? Or recommendations on other methods through which I can try accomplish this task?

Thank you very much for your time!

2

Are you able to create tensors on each device?
Could you run this code and check the device ids?

x = torch.randn(10)
for id in range(4):
    y = x.to(id)
    print(y.device)

Also, note that the device ids start with 0, in case you would like to use the “first two” GPUs.