[Solved, Self Implementing] How to return sparse tensor from nn.Embedding()

Hi.

I want to get the sparse tensor when LongTensor is passed to Embedding Layer.

I want to get adjacency matrix as shape (N,N), and it has value at (i,j) from Embedding Layer.
There are a lot of zeros in the adjacency matrix, so I want to get sparse tensor.

how to do that?

1 Like

I made it.

Assumption

The input is LongTensor which contains indices for embedding as shape has (batchsize, N * N).
In default, embedded value is 0 if index is 0. If you want to change this index, change “padding_idx” any number.
The output is adjacency sparse matrix as shape has (batchsize, N, N).

https://github.com/shllln/SparseEmbedding

thank you.

I want to do the exact same thing. However, the repository is not available anymore…any ideas how to do this efficiently?