How to calculate two tensors with different shape?

1

Sorry about post so simple question, I’m a freshman with PyTorch.

I need to normalize a feature tensor at inference stage.

Every average and STD values of every columns have been calculated already when training.

Now I get a row of new data to be inferred, and I need to normalize them at first.
But the shape of input is difference to the average tensor(same for STD), I can’t finger it out how to make that.

For example:
The average tensor is 1 dimension, and each element in it is for each column of the feature tensor in same order. The input has 2 dimensions.

avg = torch.rand(5)
input = torch.rand(3, 5)

I know that can be done with a loop, but it’s not pythonic.
I guess that there must be a solution in vector way.

Further more, when the input has 3 dimensions, i.e. [batch_size, row_count, column_count], how do it?

avg = torch.rand(5)
input = torch.rand(1, 3, 5)

Thanks!

2

You can unsqueeze the smaller tensor explicitly and/or allow broadcasting to apply the operation elementwise:

avg = torch.rand(5)
input = torch.rand(1, 3, 5)

input - avg
# tensor([[[ 0.0108,  0.4138,  0.0683,  0.1598, -0.1582],
#         [ 0.5192,  0.3414, -0.3722,  0.8293,  0.1087],
#         [-0.1535, -0.0606,  0.5320,  0.1019, -0.2201]]])

input - avg[None, None, :]
# tensor([[[ 0.0108,  0.4138,  0.0683,  0.1598, -0.1582],
#         [ 0.5192,  0.3414, -0.3722,  0.8293,  0.1087],
#         [-0.1535, -0.0606,  0.5320,  0.1019, -0.2201]]])