Is it normal for compiled model outputs to have different tensor strides?

torch.compile
1

Hello everyone,

I recently compiled my model using torch.compile, and I noticed that the results after compilation have some layout differences compared to the original (eager) execution.

When I check the outputs using torch.testing.assert_close, I sometimes get messages like:

The values for attribute 'stride()' do not match: (27, 1, 9, 3) != (27, 9, 3, 1)

or

The values for attribute 'stride()' do not match: (1, 2) != (2, 1)

The numerical values themselves appear identical, but the stride orders differ.
I’d like to ask: is this expected or normal behavior after model compilation?
Does the compiler sometimes change the memory layout (e.g., due to optimization or contiguous conversion)?

Thanks a lot for your time and for maintaining this great project!

2

Yes, this is expected. From the missing manual:

Stride divergence
PT2 is not guaranteed to produce tensors with exactly the same stride as the eager variants. This can potentially cause correctness problems (or runtime crashes) outside of the torch.compile region if subsequent code is expecting a specific striding without testing, or is using the limited subset of APIs (mostly torch.as_strided, and also obscure situations involving mutating through .reshape()/.contiguous()) which are sensitive to input of strides.

TODO: There is supposed to be a way to fix strides, similar to how we can do this for custom ops.

1 Like
3

That’s normal. torch.compile can change the tensor layout sometimes for speed, the values themselves stay fine

4

Thanks a lot for the clarification! That makes sense — good to know it’s expected behavior with torch.compile. Appreciate the detailed explanation!