I have two tensors of shape (16, 300) and (16, 300) where 16 is the batch size and 300 is some representation vector. I want to compute the element-wise batch matrix multiplication to produce a matrix (2d tensor) whose dimension will be (16, 300). So, in short I want to do 16 element-wise multiplication of two 1d-tensors.
I can do this using a for loop but is there any way, I can do it using torch API?
If you want elementwise multiplication, use the multiplication operator (
*); if you want batched matrix multiplication use
torch.bmm does matrix multiplication, not element-wise multiplication, so it can’t fulfill my purpose. (*) operator with a for loop is working for me. Btw, I also tried
torch.cmul() but it is not working. Can you tell me why? Its giving me the error -
AttributeError: module 'torch' has no attribute 'cmul'. What I am missing?
What is batch element-wise multiplication? Can you show me your for loop?
there is no
torch.cmul in pytorch, not sure why/where you picked it up or why you expect it to be there.
If you have tensor a and b both of shape (16, 300) and you want to get a tensor of shape (16, 300) by element-wise multiplication, I assume you can do a*b?
That is basically what
torch.mul do. You may safely use it.
Thank you everyone.
torch.mul worked for me. I am getting (16, 300) tensor from the two tensors of shape (16, 300) using
You should be able to get the same result just doing
a * b (this expression expands to the same thing as
torch.mul(a, b) or
How can I do element-wise mul between a matrix and a vector, say
3x5 mat and
3x1 vec, as in numpy? Pytorch’s
torch.mul told me that the sizes are inconsistent.
mat @ vec will also work in Py3.5+.
mat @ vec is not what I am looking for. I want element-wise multipication with auto broadcasting. @jekbradbury
we dont have auto-broadcasting yet.
Could we use just
A * B
if both of them are matrices
A * v.expand_as(A)
if A is matrix and v is a vector ?
It worked for me but i am not sure if the BP works fine.
I find that
torch.mul is an overloaded function since
b can be a tensor or value. Is
a = a.mul(b) an in-place operation when
b is a value?
I solved like this, but maybe I am late to the party:
def bmul(vec, mat, axis=0):
mat = mat.transpose(axis, -1)
return (mat * vec.expand_as(mat)).transpose(axis, -1)