How to know on which GPU the tensor is?

I have multiple GPUs and im using dataparallel.
Every time i print my tensor in my model it shows that it is running on one of the GPUs (e.g. GPU 0 or GPU 1, or GPU 2)
is there a way to get the gpu index that the tensor is using at each time(i.e. 0, or 1 or 2)?

You can call .device.index on your tensor:

x = torch.randn(1, device='cuda')
device_id = x.device.index
1 Like

Thank you so much
is there a way to do it in old version of pytorch (0.3.0)?

I’m not completely sure, but I think a torch.cuda.*Tensor will have a get_device method.
As far as I remember, CPU tensors won’t have such a method, so you should check, if the tensor is on the GPU (or if the method exists) first.