How to copy cuda memory to or from cuda tensor?

C++
1

I create a cuda tensor use code like below:

auto my_tensor = torch::ones({1,3,512,512},torch::device(torch::kCUDA,0));

so how can I copy the data in a cuda memory to a cuda tensor ,or copy from cuda tensor to cuda memory directly? What I want is to be able to complete the copy inside GPU without having to do GPU->CPU->GPU copy.

2

Can I use cudaMemcpy() directly?

3

I’m not sure what you want to do here, but there are several ways to do things:

  • use from_blob with a pointer to cuda memory,
  • create the cuda tensor and use the tensor.data_ptr() as a destination,
  • create a cuda tensor and use tensor.copy_(src)
4

thanks tom,you said I can create a cuda tensor and use the tensor.data_ptr() as a destination, so can I be sure the return of datr_ptr() from a cuda tensor is a cuda memory address ?

I had tried to use the data_ptr() of a created cuda tensor as the destination or source pointer of cudaMemcpy() before, it works well, but I’m not sure it is valid;

5

Yes, this is what PyTorch uses internally, too.

1 Like