Few questions about TensorView - Reshape difference

While analyzing inferMixedNet unit test with OpenCL backend I noticed something strange/unexpected and I’d appreciate some explanation.

Generated low-level IR looks as follows (code part only)

code {
  0 %tr_res = allocactivation  { Ty: float<2 x 16 x 16 x 3>} // size: 6144 // Users: @in 2, @out 5, @out 1
  1 %tr = transpose @out %tr_res, @in %var { Shuffle: [0, 2, 3, 1]}
  2 %tr_res2 = tensorview @in %tr_res { Ty: float<2 x 768>, Offsets: [0, 0, 0, 0]} // Users: @in 4
  3 %fc_add_bias_res = allocactivation  { Ty: float<2 x 16>} // size: 128 // Users: @in 6, @out 4, @in 10, @out 9, @out 11, @in 9, @in 8, @out 6
  4 %fc_dot = matmul @out %fc_add_bias_res, @in %tr_res2, @in %weights
  5 %dealloc = deallocactivation @out %tr_res // size: 6144
  6 %fc_add_bias = batchedadd @out %fc_add_bias_res, @in %fc_add_bias_res, @in %bias
  7 %tanh_res = allocactivation  { Ty: float<2 x 16>} // size: 128 // Users: @in 13, @out 10, @out 14, @in 10, @out 8
  8 %tanh = tanh @out %tanh_res, @in %fc_add_bias_res
  9 %sig = sigmoid @out %fc_add_bias_res, @in %fc_add_bias_res
  10 %add = elementadd @out %tanh_res, @in %tanh_res, @in %fc_add_bias_res
  11 %dealloc3 = deallocactivation @out %fc_add_bias_res // size: 128
  12 %fc_dot1_res = allocactivation  { Ty: float<2 x 16>} // size: 128 // Users: @in 16, @out 15, @out 17, @in 15, @out 13
  13 %fc_dot1 = matmul @out %fc_dot1_res, @in %tanh_res, @in %weights1
  14 %dealloc4 = deallocactivation @out %tanh_res // size: 128
  15 %fc_add_bias1 = batchedadd @out %fc_dot1_res, @in %fc_dot1_res, @in %bias1
  16 %SM = softmax @out %ret, @in %fc_dot1_res
  17 %dealloc7 = deallocactivation @out %fc_dot1_res // size: 128
}

I have problems in understanding what TensorView operation is doing (at line #2)?
In OpenCL backend sources it is only used in allocateMemory, because during execute() loop it is nop. Also, the code in allocateMemory is not clear when it comes to this operation tbh.

From what I see in IR it looks like this operation is behaving as reshape only changing the size of input. However, this introduced much more questions. Why not just use reshape instruction? What’s even more confusing is the fact that OCL backend in execute() doesn’t have Reshape. Actually, even there is no ReshapeInst (at least I couldn’t spot it in ClassGen files - only node). To add to this - dot file with this network has Reshape block in place where TensorView should be :slight_smile:

I noticed same behavior even in cases where createReshape is explicitly written, e.g. in inferComplexNet1. Instead of Reshape instruction in IR I see TensorView instructions.

To summarize, what I’d like to understand:

  1. What TensorView instruction is doing?

  2. What is the difference between TensorView and Reshape?

  3. How I can get the output size of TensorView? In other words, how I can tell to what dimensions I should reshape my Tensor. Should it be something like TV->getTy()->getDims()?

  4. TensorView operation has only one operand - @in and no @out, yet in next instructions (in example above line #4) it is used as an input. How I can take the name of the output? Normally I’d do something like TV->getDest()->getName(), but here it’s not possible. Any suggestion?

Thanks for support!

2 Likes

Good questions. You’re exactly right that TensorView is behaving as a reshape (in fact, if you look at IRGen for Reshape you can see that we generate TensorView for Reshape).

TensorView is a generic IR instruction that represents taking a “view” into a memory region of a tensor without creating any new allocation. Reshaping is one example (change in type without change in data), another is slicing (looking into a subregion without copying the data). Since it deals with underlying memory it doesn’t exist at the Graph level, only the IR level, which is why you see Reshape in the dot files but not in the IR.

I’m not actually sure about getting the view size or “output” name of a TV. I’d probably look at the generated class implementation in AutoGenInstr.h . I also kind of agree the way we use @out params and names on the RHS in the IR is pretty confusing :slight_smile:

1 Like

Hi Sebastian – just to be super clear, as you noticed there is no Reshape Instruction. Reshape Nodes are always implemented via Tensorview Instructions during IRGen, as Bert noted/linked to. IRGen generates Instructions (low-level) IR from the Node (high-level) IR.

Another example where you would see one Node implemented via different Instructions is the Concat node (see the IRGen case here). A Concat is implemented as one or more InsertTensor Instructions. There is no Concat Instruction.

For getting the shape and name, the Instruction class derives from Value, and Value derives from both Named and Typed. So for the name, you should be able to do TV->getName(). And for the shape, you should be able to do TV->dims(). There should be example of both of these in use in lib/Optimizer/IROptimizer.cpp.

1 Like

Thanks for info. It helped a lot!

torch.view has existed for a long time. It will return a tensor with the new shape. The returned tensor will share the underling data with the original tensor. See the documentation here.

On the other hand, it seems that torch.reshape . this method will

Return a tensor with the same data and number of elements as input, but with the specified shape. When possible, the returned tensor will be a view of input. Otherwise, it will be a copy. Contiguous inputs and inputs with compatible strides can be reshaped without copying, but you should not depend on the copying vs. viewing behavior.

It means that torch.reshape may return a copy or a view of the original tensor. You can not count on that to return a view or a copy. According to the developer:

if you need a copy use clone if you need the same storage use view. The semantics of reshape are that it may or may not share the storage and you don’t know beforehand.

1 Like