TypeError: new() received an invalid combination of arguments - got (Tensor, Tensor)

muhibkhan · September 5, 2019, 6:45am

Hey mates. Hope you are in good health. I need some guidance. I’m rebuilding an embedding model from TensorFlow to PyTorch. Here is the link:

github.com

Sujit-O/pykg2vec/blob/702e0d4012195dd8b807ec9f412887a7f3c02c9e/pykg2vec/core/TransM.py#L124


      
          
              score_pos = pos_r_theta*self.distance(pos_h_e, pos_r_e, pos_t_e)
              score_neg = neg_r_theta*self.distance(neg_h_e, neg_r_e, neg_t_e)
          
              self.loss = tf.reduce_sum(tf.maximum(score_pos + self.config.margin - score_neg, 0))
          
          def test_batch(self):
              """Function that performs batch testing for the algorithm.
          
                 Returns:
                     Tensors: Returns ranks of head and tail.
              """
              head_vec, rel_vec, tail_vec = self.embed(self.test_h_batch, self.test_r_batch, self.test_t_batch)
          
              norm_ent_embeddings = tf.nn.l2_normalize(self.ent_embeddings, axis=1)
              score_head = self.distance(norm_ent_embeddings, tf.expand_dims(rel_vec, 1), tf.expand_dims(tail_vec, 1))
              score_tail = self.distance(tf.expand_dims(head_vec, 1), tf.expand_dims(rel_vec, 1), norm_ent_embeddings)
          
              _, head_rank = tf.nn.top_k(score_head, k=self.data_stats.tot_entity)
              _, tail_rank = tf.nn.top_k(score_tail, k=self.data_stats.tot_entity)

On line 112 and 113, As I build and pass in my pytorch code:

    pos_r_theta = nn.Embedding(self.theta, pos_r)
    neg_r_theta = nn.Embedding(self.theta, neg_r)

It show me error (attached image). Q1: why this embedding not accepting the tensors as my parameters? Q2: what is the difference between tf.nn.embedding_lookup() in tf and nn.embedding() in pytorch? Q3: Does pytorch support any function similar to np.asarray (code line 103) (now want to use torch.from_numpy, proceed only in pytorch)?

spanev · September 5, 2019, 9:58am

Hey @muhibkhan,

A1: When you do this:

pos_r_theta = nn.Embedding(self.theta, pos_r)

you are creating an embedding laye where the two first args (self.theta, pos_r) should be integers indicating first the number of embeddings, then the embeddings dim.
Here’s an example of how to use it:

# in __init__
num_embs = 3
emb_dim = 10
emb = nn.Embedding(num_embs, emb_dim) 
...
# later in forward
x = torch.LongTensor([2,1,0])

and the documentation

A2: From what I see from tf.nn.embedding_lookup, they are basically the same thing.

The difference being that the lookup table initialization is automatic in PyTorch (based on the arguments the num_embeddings x embedding_dim param will be initialized from a standard normal distribution), while you have to pass the already initialized table in TF (the param argument).

A3: If I understand correctly you want to create a torch.Tensor from a list, similarly to what is done on line 103.

You can simply create the Tensor by passing your array list to the constructor:

list = [1,2,3]
tensor = torch.FloatTensor(list) 
#or
tensor = torch.LongTensor(list)

Hope that answers some of your questions.

muhibkhan · September 5, 2019, 10:32am

Thanks mate. In attached link, kindly see lines (58, 59, 103) that used by the lines 112 & 113. He passed the tensor as parameters. When IO passed the tensor, it showed me error of - got (Tensor, Tensor)…?