Asterisk with torch.cuda.FloatTensor

1

In numpy, * operator is element wise multiplication (similar to the Hadamard product for arrays of the same dimension), not matrix multiply as per this.

When I used * operation with two torch.cuda.FloatTensor A and B , results torch.cuda.FloatTensor C

>>> A.size()
(131072, 3)
>>> B.size()
(131072, 1) 
>>> C = A * B 
>>> C.size()
(131072, 3)

What operation is happening here between A and B ?

  1. not matrix multiplication A.size(1) not equal B.size(0)

  2. not element wise multiplication A.size() not equal B.size()

2

PyTorch supports numpy’s broadcasting. Have a look at this explanation.
Numpy should show the same behavior:

a = np.random.randn(10, 3)
b = np.random.randn(10, 1)
c = a * b
print(c.shape)
>> (10, 3)