Equivalent of tf.embedding_lookup()

Let’s say i have this code in tf. I want to rewrite it in pytorch

self.W = tf.v1.get_variable("W",dtype=tf.v1.float32,
                                            initializer=tf.v1.constant(self.params['wordvectors'].astype('float32')))
self.input_support_set_sents = tf.v1.placeholder(tf.v1.int32,
                                                         [None, None, sen_len],
                                                         'support_set_sents')
self.support_set_sents = tf.v1.nn.embedding_lookup(self.W, self.input_support_set_sents)

So far i have this:

  self.W = torch.tensor(config['wordvectors'], dtype=torch.float32)
   # input_support_set_sents passed by arg
  self.support_set_sents = None

embedding_lookup() in tf basically takes all words from second parameter and returns their emedding valeus from first argument.

1 Like

Based on the description I would assume you can simply index your embedding matrix with the word indices.

Can you provide code for that?

This would create an embedding and use x to get the corresponding embedding vector at index 0:

emb = nn.Embedding(10, 100)
x = torch.tensor([0])
out = emb(x)

But i already have my embeddings stored in W.
Also my problem is about placeholders in tf. I understand difference in this frameworks but i don’t really know how to rewrite it in pytorch

In that case you can directly index the weight matrix:

emb = nn.Embedding(10, 100)
x = torch.tensor([0])
out = emb(x)
out_manual = emb.weight[x, :]
print((out_manual == out).all())
> tensor(True)

I’m not 100% sure, what placeholders in TF are, but aren’t they used to feed data into the graph?

https://pytorch.org/docs/stable/torch.html#torch.index_select

2 Likes

https://pytorch.org/docs/stable/generated/torch.index_select.html#torch.index_select

1 Like