The substraction of different size Tensor

Hi everyone,
I’m a little confused with the substraction of the Tensor with different size, e.g. the Tensor(1, 3, 2) - Tensor(3, 1, 2).

import torch
import torch.nn as nn

tensor1 = torch.Tensor([[1, 2], [1, 2], [1, 2]])
tensor2 = torch.Tensor([[1, 1], [1, 1], [1, 1]])
a = tensor1.unsqueeze(0)
b = tensor2.unsqueeze(1)
result = a - b

After I excuted this code, the result says its size is (3, 3, 2), and the output of the substraction is just like each row of a minus the each dimension of b. However, I can’t undestand how this work. The a, in my mind, means the flat in the 3-d space. But the b is not the flat in this 3-d space. So, how could these two tensor work together?
Sorry about this stupid question.

Hi, it is called broadcast.

Here is Numpy broadcast.
Here is PyTorch broadcast.

Two tensors are “broadcastable” if the following rules hold:

  • Each tensor has at least one dimension.
  • When iterating over the dimension sizes, starting at the trailing dimension, the dimension sizes must either be equal, one of them is 1, or one of them does not exist.

Hi Eta_C,
Thanks for your reply. Maybe I should look through the tutorials first. Thanks for your helping. Having a good day~