'lengths' argument should be a 1D CPU int64 tensor, but got 0D cpu Long tensor

nlp
1

Hello,

I use torch 1.8.1+cuda10.2+torchtext0.9.1
Platform: Windows 10
Device: GeForce GTX 1050

This code

packed_embedded = nn.utils.rnn.pack_padded_sequence(input=embedded, lengths=text_lengths)

raises the error: “‘lengths’ argument should be a 1D CPU int64 tensor, but got 0D cpu Long tensor.”

I tried the following approaches:

#approach no 1:
text_lengths = torch.as_tensor(text_lengths, dtype=torch.int64, device='cpu')
packed_embedded = nn.utils.rnn.pack_padded_sequence(embedded, text_lengths)

#approach no 2:
lengths=text_lengths.cpu()

#approach no 3:
packed_embedded = nn.utils.rnn.pack_padded_sequence(embedded, torch.as_tensor(text_lengths.cpu(), dtype=torch.int64))

#approach no 4:
packed_embedded = nn.utils.rnn.pack_padded_sequence(embedded, text_lengths.type(torch.FloatTensor))

but without success.

I checked the value of

text_lengths.type()

in train() and the model and all the time it has the value of

torch.LongTensor

, no matter what I tried to change it.
Is there any other workaround for this problem?

thank you

2

What do you have as output for text_lengths.shape.

3

Hi,
@pascal_notsawo in train method I have

text, labels, text_lengths = train_batch

and the shape is text_lengths.shape → torch.Size([])

4

That’s why there is an error.
The lengths parameter of pack_padded_sequence just specifies the length of each sequence of your batch.
Documentation: torch.nn.utils.rnn.pack_padded_sequence — PyTorch 1.9.0 documentation

For an input of shape (T, B, E), or (B, T, E) if batch_first is True, where T is the length of the longest sequence (equal to lengths[0] if enforce_sorted is True), B is the batch size, and E is any number of dimensions (including 0, but in general it is the embedding dimension).
So lengths contains the length of the B sequences : it must be of shape (B), and not of empty shape as in your case. This is what causes the problem.

For example if I have the three (B=3) following sequences (here their indexes in the vocabulary after tokenisation, padding_token_index = 0) : [[2, 5, 0, 0], [1, 0, 0, 0], [7, 5, 2, 2]] . Then T = 4 and lengths = [2, 1, 4] (should be a 1D CPU int64 tensor).

5

Did you ever solve this problem. I’m having the same issue.

6

I solve the problem, sentence_length contain two dim, first dim present batch_size, second dim present each length of sentence. Hope can help you in the answer.