UCF101 Torchvision Dataset-to-DataLoader: Expected object of scalar type Float but got scalar type Short for sequence element 1

Hi, when I try to iterate on a dataloader object constructed from a UCF101 dataset object (from torchvision) as follows:

import torch #TODO: only load in needed parts of the torch library for the module
from torchvision.datasets import UCF101 #Use pre-made torchvision dataloaders for UCF101

def ucf101_dataset(root='../data/UCFvideo', annotation_path='../data/UCFlabels', frames_per_clip=16, step_between_clips=8, **kwargs):

    ucf_dataset = UCF101(root, annotation_path, frames_per_clip, step_between_clips, **kwargs)
    return ucf_dataset
    
def ucf101_dataloader(dataset, batch_size=4, shuffle=True, num_workers=1):

    return DataLoader(dataset=dataset, batch_size=batch_size, shuffle=shuffle, num_workers=num_workers)

test_set = ucf101_dataloader(ucf101_dataset(train=False))

test_set = iter(test_set)
test_set.next()

I get the following error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-14-32a0d4e18597> in <module>
      2 
      3 test_set = iter(test_set)
----> 4 test_set.next()

~/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py in __next__(self)
    817             else:
    818                 del self._task_info[idx]
--> 819                 return self._process_data(data)
    820 
    821     next = __next__  # Python 2 compatibility

~/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py in _process_data(self, data)
    844         self._try_put_index()
    845         if isinstance(data, ExceptionWrapper):
--> 846             data.reraise()
    847         return data
    848 

~/.local/lib/python3.6/site-packages/torch/_utils.py in reraise(self)
    383             # (https://bugs.python.org/issue2651), so we work around it.
    384             msg = KeyErrorMessage(msg)
--> 385         raise self.exc_type(msg)

RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 79, in default_collate
    return [default_collate(samples) for samples in transposed]
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 79, in <listcomp>
    return [default_collate(samples) for samples in transposed]
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 55, in default_collate
    return torch.stack(batch, 0, out=out)
RuntimeError: Expected object of scalar type Float but got scalar type Short for sequence element 1 in sequence argument at position #1 'tensors'

Been working on this for a while now and really don’t understand what I’ve done wrong, especially as I haven’t done much.

Any help appreciated. :slight_smile:

I don’t have the dataset locally at the moment, but could you try to pass transform=transforms.ToTensor() to the dataset initialization?

1 Like

Sure can. That gives us:

TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/user/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/user/.local/lib/python3.6/site-packages/torchvision/datasets/ucf101.py", line 105, in __getitem__
    video = self.transform(video)
  File "/home/user/.local/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 101, in __call__
    return F.to_tensor(pic)
  File "/home/user/.local/lib/python3.6/site-packages/torchvision/transforms/functional.py", line 54, in to_tensor
    raise TypeError('pic should be PIL Image or ndarray. Got {}'.format(type(pic)))
TypeError: pic should be PIL Image or ndarray. Got <class 'torch.Tensor'>

I notice the UCF101 dataset causes this warning to be thrown by torchvision’s video.py:

/home/user/.local/lib/python3.6/site-packages/torchvision/io/video.py:107: UserWarning: The pts_unit 'pts' gives wrong results and will be removed in a follow-up version. Please use pts_unit 'sec'.
  "follow-up version. Please use pts_unit 'sec'.")

Maybe this default could be causing an issue?

Found a solution of a kind. Setting shuffle to “False” in the DataLoader initialisation stops this error from occurring.

Will try digging a little deeper. If this stokes any ideas let me know. Think this should work now but interested in why exactly it doesn’t work if shuffle = True.

We now have:

def ucf101_dataloader(dataset, batch_size=4, shuffle=False, num_workers=1):

    return DataLoader(dataset=dataset, batch_size=batch_size, shuffle=shuffle, num_workers=num_workers)

test_set = ucf101_dataloader(ucf101_dataset(train=False))

test_set = iter(test_set)
print(test_set.next())

Producing lovely normal tensors:

[tensor([[[[[64, 65, 88],
            [64, 65, 88],
            [65, 66, 89],
            ...,

One thing I notice immediately is that elements of these tensors do not appear to be floats.