Is the memory of storage allocated by ```THCudaTensor_resize4d``` contiguous?

1

Hello. Is the memory of storage allocated by THCudaTensor_resize4d contiguous?

2

If it allocates, then yes. This is quite deep into low level of the library. May I ask why you want to know this?

3

@SimonW
I need to write a custom op using by cffi extension using cuda… I’ve read someone’s code about cffi extension. In his code:

with torch.cuda.device_of(input1):
            output = input1.new()
            custom_extension.forward_cuda(input1, output, .......)

I think he new a object or a struct output in python first, them in the c file the code:

THCudaTensor_resize4d(state, output, batchSize, nOutputChannels, outputHeight, outputwidth);

I think here he allocate the storage of output in c code.
Then in cuda kernel, he do the thing on tensor ouput as it’s memory is continuous…

I have wrote another custom op yet. My solution to deal with the problem of contiguous storge is
to invoke output = output.contiguous() in python.
However here the tensor’s storage is allocated in c code, therefore I am not sure whether the storage
return by resize4d is contiguous…

1 Like
4

Be careful, if I’m not mistaken, if the size matches exactly, it doesn’t allocate new memory, the returned tensor may not be contiguous.