Tensors are on different GPUS

21

For:
.cuda(device_id=gpus[0])

what do you mean by gpus? obviously it throws an undefined error.

22

gpus is a list of gpu ids you want to use, i.e. gpus=[1, 3].

23

I am also wondering if this issue is related to the problem I just posted here : [ Solved] nn.DataParallel with ModuleList of custom modules fails on Multiple GPUs

24

yes in my case, tensors were placed on gpu but net was not placed on cuda so i got this error.

25

Hi, i also encounter this error, did you find any solution? thanks~

1 Like
27

Hi, can you please help me with how to use the nn.ModuleList in order to get around the following error “RuntimeError: tensors are on different GPUs”

28

Hi @smth, I am facing this issue, I have a custom layer, it removes the standardization of the output, basically performs x*std+mean (opposite of standardization). So the tensors std and mean, are class variables for this layer, an error is thrown when the input passes to this layer - ‘RuntimeError: binary_op(): expected both inputs to be on same device, but input a is on cuda:1 and input b is on cuda:0’.
Do I need to copy both tensors to gpu:0 or is there another way?
Thanks.

29

did you register mean and std as buffers using .register_buffer? that will help move them to GPU-x when wrapped in a DataParallel, otherwise PyTorch wouldn’t know that they have to be moved.

Reference: https://pytorch.org/docs/stable/nn.html#torch.nn.Module.register_buffer

1 Like
30

I missed this. So this is the correct way to have variables in custom layers. Thanks @smth, it works.