# How can I do element-wise batch matrix multiplication?

I have two tensors of shape (16, 300) and (16, 300) where 16 is the batch size and 300 is some representation vector. I want to compute the element-wise batch matrix multiplication to produce a matrix (2d tensor) whose dimension will be (16, 300). So, in short I want to do 16 element-wise multiplication of two 1d-tensors.

I can do this using a for loop but is there any way, I can do it using torch API?

2 Likes

If you want elementwise multiplication, use the multiplication operator (`*`); if you want batched matrix multiplication use `torch.bmm`.

7 Likes

`torch.bmm` does matrix multiplication, not element-wise multiplication, so it can’t fulfill my purpose. (*) operator with a for loop is working for me. Btw, I also tried `torch.cmul()` but it is not working. Can you tell me why? Its giving me the error - `AttributeError: module 'torch' has no attribute 'cmul'`. What I am missing?

What is batch element-wise multiplication? Can you show me your for loop?

there is no `torch.cmul` in pytorch, not sure why/where you picked it up or why you expect it to be there.

1 Like

If you have tensor a and b both of shape (16, 300) and you want to get a tensor of shape (16, 300) by element-wise multiplication, I assume you can do a*b?

1 Like

That is basically what `torch.mul` do. You may safely use it.

2 Likes

Thank you everyone. `torch.mul` worked for me. I am getting (16, 300) tensor from the two tensors of shape (16, 300) using `torch.mul`.

You should be able to get the same result just doing `a * b` (this expression expands to the same thing as `torch.mul(a, b)` or `a.mul(b)`).

9 Likes

How can I do element-wise mul between a matrix and a vector, say `3x5` mat and `3x1` vec, as in numpy? Pytorch’s `torch.mul` told me that the sizes are inconsistent.

(solved by `mat*vec.expand_as(mat)`)

2 Likes

`mat @ vec` will also work in Py3.5+.

`mat @ vec` is not what I am looking for. I want element-wise multipication with auto broadcasting. @jekbradbury

1 Like

Could we use just

A * B
if both of them are matrices

and
A * v.expand_as(A)

if A is matrix and v is a vector ?

It worked for me but i am not sure if the BP works fine.

I find that `torch.mul` is an overloaded function since `b` can be a tensor or value. Is `a = a.mul(b)` an in-place operation when `b` is a value?

I solved like this, but maybe I am late to the party:

``````def bmul(vec, mat, axis=0):
mat = mat.transpose(axis, -1)
return (mat * vec.expand_as(mat)).transpose(axis, -1)
``````