How can I do element-wise batch matrix multiplication?

1

I have two tensors of shape (16, 300) and (16, 300) where 16 is the batch size and 300 is some representation vector. I want to compute the element-wise batch matrix multiplication to produce a matrix (2d tensor) whose dimension will be (16, 300). So, in short I want to do 16 element-wise multiplication of two 1d-tensors.

I can do this using a for loop but is there any way, I can do it using torch API?

3 Likes
2

If you want elementwise multiplication, use the multiplication operator (*); if you want batched matrix multiplication use torch.bmm.

7 Likes
3

torch.bmm does matrix multiplication, not element-wise multiplication, so it can’t fulfill my purpose. (*) operator with a for loop is working for me. Btw, I also tried torch.cmul() but it is not working. Can you tell me why? Its giving me the error - AttributeError: module 'torch' has no attribute 'cmul'. What I am missing?

4

What is batch element-wise multiplication? Can you show me your for loop?

there is no torch.cmul in pytorch, not sure why/where you picked it up or why you expect it to be there.

1 Like
5

If you have tensor a and b both of shape (16, 300) and you want to get a tensor of shape (16, 300) by element-wise multiplication, I assume you can do a*b?

1 Like
6

That is basically what torch.mul do. You may safely use it.

2 Likes
7

Thank you everyone. torch.mul worked for me. I am getting (16, 300) tensor from the two tensors of shape (16, 300) using torch.mul.

8

You should be able to get the same result just doing a * b (this expression expands to the same thing as torch.mul(a, b) or a.mul(b)).

8 Likes
9

How can I do element-wise mul between a matrix and a vector, say 3x5 mat and 3x1 vec, as in numpy? Pytorch’s torch.mul told me that the sizes are inconsistent.

(solved by mat*vec.expand_as(mat))

3 Likes
10

mat @ vec will also work in Py3.5+.

11

mat @ vec is not what I am looking for. I want element-wise multipication with auto broadcasting. @jekbradbury

12

we dont have auto-broadcasting yet.

1 Like
13

Could we use just

A * B
if both of them are matrices

and
A * v.expand_as(A)

if A is matrix and v is a vector ?

It worked for me but i am not sure if the BP works fine.

14

I find that torch.mul is an overloaded function since b can be a tensor or value. Is a = a.mul(b) an in-place operation when b is a value?

15

I solved like this, but maybe I am late to the party:

def bmul(vec, mat, axis=0):
    mat = mat.transpose(axis, -1)
    return (mat * vec.expand_as(mat)).transpose(axis, -1)
1 Like