Hello,
I have a 3-D weight tensor W with dimensionality (C,H,H) and 2-D input x (C,H).
My goal is to calculate a sum. \sum_i W_i * x_i and have an output (C,H).
How I can implement it in pytorch? Could not make it work with torch.mul
Could you elaborate a little more on what you want?
What is the size of W_i? Wouldn’t it be (H, H)? This doesn’t lead to the correct output size you want so that’s probably not the case. einsum probably does what you want, though: torch — PyTorch 2.1 documentation
Yes, its (H,H) and x_i would be (H,1) so multiplication should work somehow. Just to add a bit of context I am trying to implement time convolution (https://arxiv.org/pdf/1803.05563.pdf, section 3.1). the goal is to take a set of vectors (seq_length,H), for each of this vectors extract context (C,H) dimensionality and apply W matrix in a way to combine this context tensor into a vector (H,). In other words, W represents weights combining context vector and is convolved (or maybe proper to say correlated) through the whole sequence.