Cannot understand tensor multiplication


I have a 3-D weight tensor W with dimensionality (C,H,H) and 2-D input x (C,H).
My goal is to calculate a sum. \sum_i W_i * x_i and have an output (C,H).

How I can implement it in pytorch? Could not make it work with torch.mul

Could you elaborate a little more on what you want?

What is the size of W_i? Wouldn’t it be (H, H)? This doesn’t lead to the correct output size you want so that’s probably not the case. einsum probably does what you want, though:

Yes, its (H,H) and x_i would be (H,1) so multiplication should work somehow. Just to add a bit of context I am trying to implement time convolution (, section 3.1). the goal is to take a set of vectors (seq_length,H), for each of this vectors extract context (C,H) dimensionality and apply W matrix in a way to combine this context tensor into a vector (H,). In other words, W represents weights combining context vector and is convolved (or maybe proper to say correlated) through the whole sequence.