When reshaping an array, NumPy avoids copies when possible by modifying the strides attribute. For example, when transposing a matrix, the order of strides is reversed, but the underlying data remains identical
Thanks for addressing it. That is indeed interesting insights.
However, what if the operation following permutations requires the data layout to be packed in certain way. For instance, in the model DeepSpeech, permutation is used right before RNN layers. If the data before permutation is column-major, and RNN input has to be column-major (as required by cudnnRNNForwardTraining), then we cannot just change the strides for permutation (data is no longer column-major if the stride of the last dimension is not 1).
Is this the case where some sort of data copying has to happen?
P.S. Proof that cudnnRNNForwardTraining requires the input to be column-major, from:
Input vectors are expected to be arranged in the column-major order so strides in xDesc should be set as follows: strideA=inputSize, strideA=1, strideA=1.
========= update ================
I realized that in cases like this, a .contiguous() function is used to copy the data into column-major form. I am looking at how PyTorch implement copy() at this moment. No outstanding questions for now