I wrote a parser using Embedding that works fine on CPU.
I have added .to(torch.float64), .to(device(‘cuda’) in the appropriate places, to get to run on GPU, in particular after the creation of an Embedding.
However, when extracting the Embedding for a sequence of indices, with pdb3, I get:
. exec(cmd, globals, locals)
File “”, line 1, in
File “/project/tools/lhrparser/examples/train.py”, line 18, in
import sys
File “/project/tools/lhrparser/examples/train.py”, line 88, in main
trainer.train(training_set=training_sentences, validation_set=test_sentences, model_path=model_path)
File “/project/tools/lhrparser/src/traininghelper.py”, line 254, in train
loss = self.train_sentence(sentence=sentence)
File “/project/tools/lhrparser/src/traininghelper.py”, line 203, in train_sentence
concatenated_embedding = self.parser.encode_embeddings(sentence=sentence)
File “/project/tools/lhrparser/src/lhrparser.py”, line 41, in encode_embeddings
pos_embedding = self.model.pos_embeddings(pos_sequence)
File “/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py”, line 491, in call
result = self.forward(*input, **kwargs)
File “/project/tools/lhrparser/src/embeddings.py”, line 45, in forward
embeds = self.embeddings(sequence)
File “/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py”, line 491, in call
result = self.forward(*input, **kwargs)
File “/usr/local/lib/python3.5/dist-packages/torch/nn/modules/sparse.py”, line 108, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File “/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py”, line 1076, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected object of type torch.cuda.LongTensor but found type torch.LongTensor for argument #3 ‘index’
Uncaught exception. Entering post mortem debugging
Running ‘cont’ or ‘step’ will restart the program
/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py(1076)embedding()
→ return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
(Pdb) weight.dtype
torch.float64
(Pdb) input
tensor([ 3, 12, 13, 11, 11, 1, 14, 0, 15, 3, 12, 14,
0, 9, 16, 2, 12, 6, 15, 12, 12, 11, 0, 14,
11, 16, 12, 13])
(Pdb) weight.shape
torch.Size([17, 50])
(Pdb) padding_idx
-1
(Pdb) scale_grad_by_freq
False
(Pdb) sparse
False
Any help appreciated.