Description of "THCudaTensor_data"

1

My understanding of THCudaTensor_data( state, tensor ) is that its general function is to retrieve the data from a pytorch’s tensor and save it as an array. However, I cannot understand the specifics.

  1. what is the state?
  2. what if the initial tensor is 2D or beyond? Does it flatten the 2D into long 1D array?
  3. is the date type of the items inside the pytorch’s tensor preserved?
  4. what is the return type, and does the array reside in global memory or stack?

Thanks!

1 Like
2

Hi,

The THCudaTensor_data function does no data retrieval. It simply returns a pointer to the beginning of where the tensor content is stored.
You should then use storage_offset, size and stride to find the offset with respect to this starting point for every element of the tensor you want to access.

The state is THC state that is given to all THC functions.
The return type is a pointer to the type contained in the tensor (like float* double* int*…) note that the data live on GPU for cuda tensor and cannot be accessed from C code.

2 Likes
3

Thanks for your reply.

From my short experience working with THCudaTensors, they seem to have identical characteristics with cudaMalloc-ed arrays. Are they actually the same? If not, what are the differences? Also, do you have URL to documentations to THCudaTensors?

4

There are similar but they are not always cudaMalloc-ed. Indeed, there is a custom allocator inside pytorch that handles the allocation of the data for Tensors.
I am not sure what you call a “cudaMalloc-ed array” but these datas contain flattened data where the storage_offset, size and stride properties tells you how to access them.

I’m afraid there is no documentation for this part of the code. It is quite old now and is getting replaced. You might want to use aten now where the whole interface is much simpler to use. And is a properly documented cpp interface :wink:

5

Oh what I was asking was if the memory allocated through THCuda package would reside in the device‘s main memory and display the same caching behavior as memory allocated via cudaMalloc.

Thanks for letting me know about A different package. I will definitely take a look. Will it be compatible with pytorch 1.0 ?

6

Yes. It is compatible with pytorch 1.0.
All the extensions (in cuda, cpp) are written using ATen.

https://pytorch.org/tutorials/advanced/cpp_extension.html

1 Like