Error with Lengths in pack_padded_sequence

1

Hi,
I am trying to set up an RNN capable of utilizing a GPU but packed_padded_sequence gives me a

RuntimeError: 'lengths' argument should be a 1D CPU int64 tensor

here is how I direct gpu computing

parser = argparse.ArgumentParser(description='Trainer')
parser.add_argument('--disable-cuda', action='store_true',
                    help='Disable CUDA')
args = parser.parse_args()
args.device = None
if not args.disable_cuda and torch.cuda.is_available():
    args.device = torch.device('cuda')
    torch.set_default_tensor_type(torch.cuda.FloatTensor)
else:
    args.device = torch.device('cpu')

here is the relevant part of the code.

def Tensor_length(track):
    """Finds the length of the non zero tensor"""
    return int(torch.nonzero(track).shape[0] / track.shape[1])
.
.
.
def forward(self, tracks, leptons):
        self.rnn.flatten_parameters()
        # list of event lengths
        n_tracks = torch.tensor([Tensor_length(tracks[i])
                                 for i in range(len(tracks))])
        sorted_n, indices = torch.sort(n_tracks, descending=True)
        sorted_tracks = tracks[indices].to(args.device)
        sorted_leptons = leptons[indices].to(args.device)
        # import pdb; pdb.set_trace()
        output, hidden = self.rnn(pack_padded_sequence(sorted_tracks,
                                                       lengths=sorted_n.cpu().numpy(),
                                                       batch_first=True)) # this gives the error

        combined_out = torch.cat((sorted_leptons, hidden[-1]), dim=1)
        out = self.fc(combined_out)  # add lepton data to the matrix
        out = self.softmax(out)
        return out, indices  # passing indices for reorganizing truth

I have tried everything from casting sorted_n to a long tensor to having it be a list, but i aways seem to get the same error.
I have not worked with pytorch on gpu before and any advice will be greatly appreciated.

Thanks!

3

It’s working with a 1D CPU LongTensor for me:

T, B, C = 100, 10, 4  # longest sequence, batch_size, feature_size
input = torch.randn(T, B, C, device='cuda')
lengths, _ = torch.sort(torch.randint(1, 100, (10,)), descending=True)
print(input.type())
> torch.cuda.FloatTensor
print(lengths.type())
> torch.LongTensor
pack_padded_sequence(input, lengths)

Could you print the type and shape of sorted_n?

4

test
test1724×688 130 KB

5

That’s weird, since lengths=sorted_n.cpu() should create a 1D cpu LongTensor.
Could you just for the sake of debugging pass lengths=torch.randint(1, 15, (100,)) and see if it’s working?

6

Selection_106
Selection_1061753×655 131 KB

7

Thanks for the info!
I just debugged it and it looks like this line of code is problematic, if you’ve set a CUDA tensor as the default tensor type.
Here is a small snippet to see that the cpu() call will be ignored, since the data will be rewrapped:

lengths = torch.as_tensor(lengths.cpu(), dtype=torch.int64)
lengths.type()
> 'torch.cuda.LongTensor'

Could you create a GitHub issue and CC me there?
Let me know, if you don’t want to do it and I’ll create the issue. :wink:

The current workaround would be to avoid setting the default device or use this small hotfix:

import torch
torch.set_default_tensor_type(torch.cuda.FloatTensor)
from torch.nn.utils.rnn import pack_padded_sequence, PackedSequence


def hotfix_pack_padded_sequence(input, lengths, batch_first=False, enforce_sorted=True):
    lengths = torch.as_tensor(lengths, dtype=torch.int64)
    lengths = lengths.cpu()
    if enforce_sorted:
        sorted_indices = None
    else:
        lengths, sorted_indices = torch.sort(lengths, descending=True)
        sorted_indices = sorted_indices.to(input.device)
        batch_dim = 0 if batch_first else 1
        input = input.index_select(batch_dim, sorted_indices)

    data, batch_sizes = \
        torch._C._VariableFunctions._pack_padded_sequence(input, lengths, batch_first)
    return PackedSequence(data, batch_sizes, sorted_indices)


T, B, C = 100, 10, 4  # longest sequence, batch_size, feature_size
input = torch.randn(T, B, C, device='cuda')
lengths, _ = torch.sort(torch.randint(1, 100, (10,)), descending=True)
print(input.type())
print(lengths.type())
pack_padded_sequence(input, lengths.cpu())  # Error due to default type set to CUDA
hotfix_pack_padded_sequence(input, lengths)
4 Likes
8

Thanks! I will do so asap.

Selection_106
Selection_1061554×254 57.4 KB

but the fix gives me this error

9

Which PyTorch version are you using?
Mine might be a bit too old (1.0.0.dev20190104).

10

i am using 1.0,0.
removing sorted_indices fixed it.
Thanks a ton!

1 Like
11


here is the issue

1 Like
12

maybe i have the same problem,i try two ways,
First:
when i spcify the function code ‘pack_padded_sequence’:

def pack_padded_sequence(input, lengths, batch_first=False, enforce_sorted=True):
    if torch._C._get_tracing_state() and not isinstance(lengths, torch.Tensor):
        warnings.warn('pack_padded_sequence has been called with a Python list of '
                      'sequence lengths. The tracer cannot track the data flow of Python '
                      'values, and it will treat them as constants, likely rendering '
                      'the trace incorrect for any other combination of lengths.',
                      stacklevel=2)
    lengths = torch.as_tensor(lengths, dtype=torch.int64)
    lengths = lengths.cpu() # 此处给pytorch提的issue解决的
    if enforce_sorted:
        sorted_indices = None
    else:
        lengths, sorted_indices = torch.sort(lengths, descending=True)
        sorted_indices = sorted_indices.to(input.device)
        batch_dim = 0 if batch_first else 1
        input = input.index_select(batch_dim, sorted_indices)

    data, batch_sizes = \
        _VF._pack_padded_sequence(input, lengths, batch_first)
    return PackedSequence(data, batch_sizes, sorted_indices, None)

when i debug my code, the type of input and lengths is all ‘‘torch.LongTensor’’ and ‘‘torch.FloatTensor’’,

企业微信截图_15903898892171
企业微信截图_159038988921711662×1360 150 KB

but it raise the error:

File “C:\Anaconda3\lib\site-packages\torch\nn\utils\rnn.py”, line 277, in pack_padded_sequence
_VF._pack_padded_sequence(input, lengths, batch_first)
RuntimeError: ‘lengths’ argument should be a 1D CPU int64 tensor

Second i try def the function ‘hotfix_pack_padded_sequence’:

def hotfix_pack_padded_sequence(input, lengths, batch_first=False, enforce_sorted=True):
    lengths = torch.as_tensor(lengths, dtype=torch.int64)
    lengths = lengths.cpu()
    if enforce_sorted:
        sorted_indices = None
    else:
        lengths, sorted_indices = torch.sort(lengths, descending=True)
        sorted_indices = sorted_indices.to(input.device)
        batch_dim = 0 if batch_first else 1
        input = input.index_select(batch_dim, sorted_indices)

    data, batch_sizes = \
        torch._C._VariableFunctions._pack_padded_sequence(input, lengths, batch_first)
    return PackedSequence(data, batch_sizes, sorted_indices)

when i debug my code, the type of input and lengths is all ‘‘torch.LongTensor’’ and ‘‘torch.FloatTensor’’,
but it raise the same error:

“D:\soft\NLP_Frame\algorithm\modules\encoder\lstm_encoder.py”, line 55, in hotfix_pack_padded_sequence
torch._C._VariableFunctions._pack_padded_sequence(input, lengths, batch_first)
RuntimeError: ‘lengths’ argument should be a 1D CPU int64 tensor

I’m going crazy， :sob:

13

bounding the packing inside a different default tensor type is a hack that I have found works

torch.set_default_tensor_type(torch.FloatTensor)
<call to pack padded sequences>
if self.device == torch.device("cuda"):
        torch.set_default_tensor_type(torch.cuda.FloatTensor)

Hopefully this works for you

15

i find the debug，because my input.shape is [6, 574, 574, 100], lengths.shape is[6, 574], so the pack_padded_sequence(input, lengths) can not run, the error is

‘lengths’ argument should be a 1D CPU int64 tensor

what can i specify it?

16

I think this is a device error rather than an error in the function. Looking at other discussion on the same exception, they are mentioning that you need to move the lengths into the CPU (which is exactly what the error says).

link to the issue in github

17

it works for me

image
image869×303 7.54 KB