The substraction of different size Tensor

#1

Hi everyone,
I’m a little confused with the substraction of the Tensor with different size, e.g. the Tensor(1, 3, 2) - Tensor(3, 1, 2).

import torch
import torch.nn as nn

tensor1 = torch.Tensor([[1, 2], [1, 2], [1, 2]])
tensor2 = torch.Tensor([[1, 1], [1, 1], [1, 1]])
a = tensor1.unsqueeze(0)
b = tensor2.unsqueeze(1)
result = a - b

After I excuted this code, the result says its size is (3, 3, 2), and the output of the substraction is just like each row of a minus the each dimension of b. However, I can’t undestand how this work. The a, in my mind, means the flat in the 3-d space. But the b is not the flat in this 3-d space. So, how could these two tensor work together?
Sorry about this stupid question.

#2

Hi, it is called broadcast.

Here is Numpy broadcast.
Here is PyTorch broadcast.

Two tensors are “broadcastable” if the following rules hold:

  • Each tensor has at least one dimension.
  • When iterating over the dimension sizes, starting at the trailing dimension, the dimension sizes must either be equal, one of them is 1, or one of them does not exist.
#3

Hi Eta_C,
Thanks for your reply. Maybe I should look through the tutorials first. Thanks for your helping. Having a good day~