# Is there a way to calculate cosine similarity between all combinations of embeddings?

**URL:** <https://discuss.pytorch.org/t/is-there-a-way-to-calculate-cosine-similarity-between-all-combinations-of-embeddings/19446>\
**Category:** Uncategorized\
**Created:** [June 9, 2018, 4:12pm UTC](https://discuss.pytorch.org/t/is-there-a-way-to-calculate-cosine-similarity-between-all-combinations-of-embeddings/19446 "2018-06-09T16:12:45Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![josmi9966](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/josmi9966/32/2272_2.png) [@josmi9966](https://discuss.pytorch.org/u/josmi9966)\
**Post date:** [June 9, 2018, 4:12pm UTC](https://discuss.pytorch.org/t/is-there-a-way-to-calculate-cosine-similarity-between-all-combinations-of-embeddings/19446/1 "2018-06-09T16:12:45Z")

</div>

If I have two tensors of shape (N, D) where D is the size of the embeddings, is there a simple, efficient  
way to calculate the tensor of shape (N, N) which contains all the similarities between any pair of the N embeddings?

(Updated to correct the result shape)

---

<div class="post-metadata">

**Author:** ![royboy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/royboy/32/4106_2.png) [@royboy](https://discuss.pytorch.org/u/royboy)\
**Post date:** [June 11, 2018, 10:59pm UTC](https://discuss.pytorch.org/t/is-there-a-way-to-calculate-cosine-similarity-between-all-combinations-of-embeddings/19446/2 "2018-06-11T22:59:42Z")

</div>

There’s no way to do it directly, but here’s a relatively low effort way to do this (just use cosine similarity instead of distance):

> [@How to write custom loss function in terms of pairwise distances of outputs?](https://discuss.pytorch.org/t/how-to-write-custom-loss-function-in-terms-of-pairwise-distances-of-outputs/18588):
>
> Hi there, I am trying to implement a loss function that depends on pairwise distances of output data. Specifically I have neural network f which take X as input and the output is $Y = {y\_i}\_{i=1}^N$. My error function has following form: $E = | d\_{ij} - c\_{ij} | ^2$ where d\_{ij} is distance between y\_i and y\_j, and c\_{ij} is the target value for this distance. I know how to write “vectorized” loss function like MSE, softmax which would take a complete vector to compute the loss. But in my …

---

<div class="post-metadata">

**Author:** ![Konpat\_Ta\_Preechakul](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/konpat_ta_preechakul/32/3311_2.png) [@Konpat\_Ta\_Preechakul](https://discuss.pytorch.org/u/Konpat_Ta_Preechakul)\
**Post date:** [June 12, 2018, 8:15am UTC](https://discuss.pytorch.org/t/is-there-a-way-to-calculate-cosine-similarity-between-all-combinations-of-embeddings/19446/3 "2018-06-12T08:15:53Z")

</div>

Similarities for any pair of N embeddings should be of shape (N, N) ? Where does the last “D” come from?

Btw, I have read that if you have embeddings A, B and normalized it in such a way that the norm of each embedding equals to 1. matmul(A, B.t()) should be the cosine similarity for each pair of the embeddings?

---

<div class="post-metadata">

**Author:** ![josmi9966](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/josmi9966/32/2272_2.png) [@josmi9966](https://discuss.pytorch.org/u/josmi9966)\
**Post date:** [June 12, 2018, 12:53pm UTC](https://discuss.pytorch.org/t/is-there-a-way-to-calculate-cosine-similarity-between-all-combinations-of-embeddings/19446/4 "2018-06-12T12:53:14Z")

</div>

Yes, sorry, that shape was wrong, corrected it.

Also, now that I think of it I think you are right and one could probably calculated this  
in a way that is specific to cosine similarity.

I guess what I really was interested in is if there is an abstract operation where you have two tensors and you get a result tensor by applying a function of two parameters to all pairs of values where the values are taken along some dimension of those tensors.

So to try this again more generally, lets say you have two tensors (N, D) and (M, D) and a function from (D,D) to K, I want an operation that produces a tensor (N,M,K) from applying the function to all the pairs of D along the first axis.

This looks like such an obvious operation that I just feel it MUST exist! 🙂

---

<div class="post-metadata">

**Author:** ![praateekmahajan](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/praateekmahajan/32/2228_2.png) [@praateekmahajan](https://discuss.pytorch.org/u/praateekmahajan)\
**Post date:** [April 2, 2020, 1:28am UTC](https://discuss.pytorch.org/t/is-there-a-way-to-calculate-cosine-similarity-between-all-combinations-of-embeddings/19446/5 "2020-04-02T01:28:22Z")

</div>

Definitely not the efficient solution you’re looking for, but this seems to come up first on google, so I’ll write down my hacky solution for others. For me this was easy and fast enough because my `num_embedding` was \< 10.

Just changing line 1 and printing(final) works.

```python
embeddings = list(model.EMBEDDING_NAME.parameters())[0].detach().cpu()

final = []
for i in range(len(embeddings)):
  ans = []
  for j in range(len(embeddings)):
  	similarity = torch.cosine_similarity(embeddings[i].view(1,-1), 
  										 embeddings[j].view(1,-1)).item()
  	ans.append(similarity)
  final.append(ans)

```

And I used a heatmap to display what I wanted so there goes that.

```auto
import seaborn as sns
fig = plt.figure()
sns.heatmap(final, annot=True)
display(fig)

```