Hey there.
So, I noticed that when I use ‘pin_memory=True’ for my Dataloaders what is returned is actually slightly different from what should be (and is, when ‘pin_memory=False’) returned.
Mostly, what I have noticed so far is that PackPadded sequences become just the tensors that form them (data and batch_sizes) and tuples get changed to lists.
This is an example of what my current dataloader should return, and is returning if ‘pin_memory=False’ (and it is what it is return at the collate function level, regardless of the ‘pin_memory’ setting):
((PackedSequence(data=tensor([[-0.3763, -0.9540, 0.1961, ..., -0.2956, 1.2341, 0.2977],
[-0.3763, -0.9540, 0.1961, ..., -0.1619, 1.2689, -0.7896],
[ 0.1915, 0.2300, -0.2894, ..., -0.8150, 1.8169, 2.0587],
...,
[ 0.2478, -0.2490, 0.0222, ..., -2.9116, 1.4836, -0.5898],
[-0.9930, -0.5766, 0.2790, ..., 0.7489, 1.2063, -0.0581],
[-0.8871, -0.2004, -1.0601, ..., 0.0524, -0.0852, 0.0805]]),
batch_sizes=tensor([2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1])),
([5, 9, 28, 47, 78, 80, 82, 84, 103, 118, 121, 131, 133, 135, 160, 212, 246, 263, 298, 310, 337, 347, 365, 399, 400, 401, 413, 320, 442, 455, 486, 496, 515, 530, 561, 568, 579, 582, 585, 587, 590, 599, 105, 625, 633, 648, 656, 701, 707, 714, 722, 568, 734, 738, 739, 741, 743, 745, 764, 785, 801, 843, 876, 895, 906, 910, 915, 932, 951, 956, 959, 981, 987, 1014, 1017, 1029, 1042, 1045, 1068, 1081],
None)),
(tensor([0, 0]),
tensor([[ 1, 2, 16, 4, 6, 17, 9, 10, 18, 19, 20, 21, 10, 22, 12, 23, 24, 6, 25, 15],
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 0, 0, 0, 0, 0]])))
and this is what it returns when ‘pin_memory=True’:
[[[tensor([[-0.3763, -0.9540, 0.1961, ..., -0.2956, 1.2341, 0.2977],
[-0.3763, -0.9540, 0.1961, ..., -0.1619, 1.2689, -0.7896],
[ 0.1915, 0.2300, -0.2894, ..., -0.8150, 1.8169, 2.0587],
...,
[ 0.2478, -0.2490, 0.0222, ..., -2.9116, 1.4836, -0.5898],
[-0.9930, -0.5766, 0.2790, ..., 0.7489, 1.2063, -0.0581],
[-0.8871, -0.2004, -1.0601, ..., 0.0524, -0.0852, 0.0805]]),
tensor([2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1])],
[[5, 9, 28, 47, 78, 80, 82, 84, 103, 118, 121, 131, 133, 135, 160, 212, 246, 263, 298, 310, 337, 347, 365, 399, 400, 401, 413, 320, 442, 455, 486, 496, 515, 530, 561, 568, 579, 582, 585, 587, 590, 599, 105, 625, 633, 648, 656, 701, 707, 714, 722, 568, 734, 738, 739, 741, 743, 745, 764, 785, 801, 843, 876, 895, 906, 910, 915, 932, 951, 956, 959, 981, 987, 1014, 1017, 1029, 1042, 1045, 1068, 1081],
None]],
[tensor([0, 0]),
tensor([[ 1, 2, 16, 4, 6, 17, 9, 10, 18, 19, 20, 21, 10, 22, 12, 23, 24, 6, 25, 15],
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 0, 0, 0, 0, 0]])]]
Is this the expected behaviour?