Get error in embedding layer

1

Hi, I have a text corpus and I have a score for each sentence of it. I want to make a RNN model to predict these scores. I have written some codes to do it. In train phase, a batch of data (sentence-score samples) is given to my classifier by:
output=classifier(input,seq_lengths)
input is a tensor that each row of it is a sequence of word embedding vectors of the words in a sentence.
seq_lengths is a list of its sentence lengths.
input is a tensor with size:
batch_size*(max_length_of_sentences*word_embedding_vector_length)

but I get an error in running this line of code:
embedded=self.embedding(input)
the error is:

Traceback (most recent call last):
  File "/home/mahsa/PycharmProjects/PyTorch_env_project/Thesis/proj2/mahsa_rnn_sent_classification.py", line 277, in <module>
    train()
  File "/home/mahsa/PycharmProjects/PyTorch_env_project/Thesis/proj2/mahsa_rnn_sent_classification.py", line 238, in train
    output = classifier(input, seq_lengths)
  File "/home/mahsa/anaconda3/envs/pytorch_env/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/mahsa/PycharmProjects/PyTorch_env_project/Thesis/proj2/mahsa_rnn_sent_classification.py", line 212, in forward
    embedded = self.embedding(input)
  File "/home/mahsa/anaconda3/envs/pytorch_env/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/mahsa/anaconda3/envs/pytorch_env/lib/python3.5/site-packages/torch/nn/modules/sparse.py", line 94, in forward
    self.scale_grad_by_freq, self.sparse
  File "/home/mahsa/anaconda3/envs/pytorch_env/lib/python3.5/site-packages/torch/nn/_functions/thnn/sparse.py", line 53, in forward
    output = torch.index_select(weight, 0, indices.view(-1))
TypeError: torch.index_select received an invalid combination of arguments - got (torch.FloatTensor, int, !torch.FloatTensor!), but expected (torch.FloatTensor source, int dim, torch.LongTensor index)

can you guide me?

2

Hi,

The problem is that the input you give to the embedding layer is not a LongTensor. You should convert your input before giving it to the embedding layer: input.long().

1 Like
3

Hi, Thanks. But my input is consisted of word embedding vectors that they are float not int.

4

From the doc the embedding layer takes as input the indices of the embeddings it should output the representation for. Isn’t that what you give as input?

5

Thanks @albanD. your doc saves me from a huge mistake. I was feeding wrong input to embedding layer. But now I received a new error in computing loss:

Traceback (most recent call last):
  File "/home/mahsa/PycharmProjects/PyTorch_env_project/Thesis/proj2/mahsa_rnn_sent_classification.py", line 271, in <module>
    train()
  File "/home/mahsa/PycharmProjects/PyTorch_env_project/Thesis/proj2/mahsa_rnn_sent_classification.py", line 234, in train
    loss = criterion(output, target)
  File "/home/mahsa/anaconda3/envs/pytorch_env/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/mahsa/anaconda3/envs/pytorch_env/lib/python3.5/site-packages/torch/nn/modules/loss.py", line 482, in forward
    self.ignore_index)
  File "/home/mahsa/anaconda3/envs/pytorch_env/lib/python3.5/site-packages/torch/nn/functional.py", line 746, in cross_entropy
    return nll_loss(log_softmax(input), target, weight, size_average, ignore_index)
  File "/home/mahsa/anaconda3/envs/pytorch_env/lib/python3.5/site-packages/torch/nn/functional.py", line 672, in nll_loss
    return _functions.thnn.NLLLoss.apply(input, target, weight, size_average, ignore_index)
  File "/home/mahsa/anaconda3/envs/pytorch_env/lib/python3.5/site-packages/torch/nn/_functions/thnn/auto.py", line 47, in forward
    output, *ctx.additional_args)
TypeError: FloatClassNLLCriterion_updateOutput received an invalid combination of arguments - got (int, torch.FloatTensor, !torch.DoubleTensor!, torch.FloatTensor, bool, NoneType, torch.FloatTensor, int), but expected (int state, torch.FloatTensor input, torch.LongTensor target, torch.FloatTensor output, bool sizeAverage, [torch.FloatTensor weights or None], torch.FloatTensor total_weight, int ignore_index)

Sorry that this problem is different from the topic but can you guide me? why my target should be torch.Long? because my target is the scores for sentences and they are float.

6

It’s actually a very similar error as the previous one. As you can see in the error message

TypeError: FloatClassNLLCriterion_updateOutput received an invalid combination of arguments - got (int, torch.FloatTensor, !torch.DoubleTensor!, torch.FloatTensor, bool, NoneType, torch.FloatTensor, int), but expected (int state, torch.FloatTensor input, torch.LongTensor target, torch.FloatTensor output, bool sizeAverage, [torch.FloatTensor weights or None], torch.FloatTensor total_weight, int ignore_index)

the function that you try to use called when doing loss = criterion(output, target) takes got as input a Float and a Double tensor (note that the DoubleTensor has some ! around it). It it was expecting Float and Long tensor. As you can see, the LongTensor is for the target.
So the second argument for your criterion that contains the target should be a LongTensor and not a DoubleTensor. Double check the doc for the criterion you used to make sure what it takes as target is what you give him.
In general in pytorch, every tensor that contains indices will be LongTensors.

7

Hi, yes I double checked my code. and the problem was solved. Thanks for valuable comments.