How to move a tensor from one gpu to another?

1

I have a tensor which is on gpu 0.
how can I move (or make a copy of it one gpu 1)?

A = torch.rand(1)
cuda0 = torch.device('cuda:0')
cuda1 = torch.device('cuda:1')
A.to(cuda0)
# now I want to move A to gpu 1
A.to(cuda1)

gives me error:

RuntimeError: CUDA error: invalid device ordinal

1 Like
2

Hi,

Are you sure the second GPU is properly available? What does device_count() returns?
You might be hidding the second GPU with CUDA_VISIBLE_DEVICES=0 as well.

1 Like
3

Hi,

yeah my bad. I missed to change the number of devices in CUDA_VISIBLE.
Thanks for pointing that out

4

Hi,

Would you tell me what happened when tensor.to is called? Is the tensor transfered directly from gpu 1 to gpu 2, or it is first transferred from gpu 1 to cpu memory and then transferred from cpu to gpu 2?

5

It depends on the hardware you have.
If possible, it will be sent directly from one GPU to the other. But not all cards support that AFAIK and it might have to go through the CPU.