Inner product 2-D weight matrix with 3-D torch of spectrograms

audio
1

Hi.

I have the following setup:
[49, 49] matrix, where each row is a probabilities vector (obtained from softmax over logits). overall it has 49 probability vectors, each with 49 examples.

[49, x, y] matrix, containig 49 spectrograms of size [x,y] each.

I try to obtain a 49 different weighted spectrograms, from each of the 49 probability vectors and 49 spectrograms.

Output size shall be [49, x, y].

I tried my best to search the net, and tried many configurtations of torch matmul, bmm, etc…

2

If I understand correctly, since you want differently weighted spectrograms with each of the 49 probability vectors and 49 spectrograms, the output size should be [49, 49, x, y]. No?

i.e., In the spectrogram size [49, x, y], batch size = 49. The same batch size as in the probability matrix of [49, 49]. For a particular spectrogram, you have 49 probability values corresponding to different classes.

3

Ok I managed to figure this out :slight_smile:
with the following:
torch.einsum(“ik,klm->ilm”, probabilities, spectrograms)

@InnovArul you undertood me wrong. Each of the rows in the probabilities matrix is a probability vector with 49 values. Now I multiply in an inner-product manner with the 49 spectrograms to get a weigted spectrogram of size [x, y]. Since the probability matrix has 49 rows I get an overall 49 weighted spectrograms so out size is [49, x, y]

I just realized einsum and this is a SUPER powerful tool! I highly recommend to any deep learning practitioner to use it