DataLoader iterator is not working for Custom Dataset

1

I am creating a custom dataset and data loader. Unfortunately, when I apply next(iter(dataloader), it gives the following error.

TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <U4

I am very new to PyTorch (I used to code in Keras). I found that this error is very common if there is no transformation for the samples. However, in my source code, I applied the following transformation for the data before returning it from getitem function.

train_transform = torchvision.transforms.Compose([
torchvision.transforms.ToTensor()
])

I also checked the shape of the sample after the transformation. It seems the shape is as it supposed to be ( torch.Size([1, 16, 132]) ), and the type is also tensor as expected ( <class ‘torch.Tensor’> ).

To confirm that, the data loader has enough items to iterate, I checked its length. It seems the count is quite accurate.

To ensure that it can handle exception automatically, I also tried below try-catch. Unfortunately, no luck yet!

dataloader_iterator = iter(train_loader)
for i in range(iterations):
try:
data, target = next(dataloader_iterator)
except StopIteration:
dataloader_iterator = iter(train_loader)
data, target = next(dataloader_iterator)

So far, my understanding is, whenever I try to use next() function on any iter object, it throws the error.
Below are the full error logs.

File “/root/dataloader.py”, line 71, in get_loader
data, target = next(dataloader_iterator)
File “/root/.virtualenvs/pytorchve/lib/python3.5/site-packages/torch/utils/data/dataloader.py”, line 345, in next
data = self._next_data()
File “/root/.virtualenvs/pytorchve/lib/python3.5/site-packages/torch/utils/data/dataloader.py”, line 385, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File “/root/.virtualenvs/pytorchve/lib/python3.5/site-packages/torch/utils/data/_utils/fetch.py”, line 47, in fetch
return self.collate_fn(data)
File “/root/.virtualenvs/pytorchve/lib/python3.5/site-packages/torch/utils/data/_utils/collate.py”, line 79, in default_collate
return [default_collate(samples) for samples in transposed]
File “/root/.virtualenvs/pytorchve/lib/python3.5/site-packages/torch/utils/data/_utils/collate.py”, line 79, in
return [default_collate(samples) for samples in transposed]
File “/root/.virtualenvs/pytorchve/lib/python3.5/site-packages/torch/utils/data/_utils/collate.py”, line 62, in default_collate
raise TypeError(default_collate_err_msg_format.format(elem.dtype))
TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <U4

Update: (adding custom dataset implementation)

class CustomDataset(Dataset):

 def __init__(self, x_data, y_data, transform=None):
     self.x_data = x_data
     self.y_data = y_data
     self.transform = transform

 def __getitem__(self, index):
     image = self.x_data[index]
     image = image[..., np.newaxis]
     label = self.y_data[index] 
     if self.transform is not None:
         image = self.transform(image)
     return image, label
2

Could you show your custom dataset implementation and specifically the __getitem__?

3

Thank you for the reply. I updated the topic description, and added custom dataset implementation code.

4

I don’t see what could be causing the issue. Could you provide some code that produces the error?

5

Thank you again. Basically, whenever I try to iterate the DataLoader object, I get this error. Following code reproduce the issue -

train_transform = torchvision.transforms.Compose([
torchvision.transforms.ToTensor()
])
train_dataset = CustomDataset(x_train, y_train, transform=train_transform)
test_dataset = CustomDataset(x_test, y_test, transform=train_transform)
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=False)
test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)
dataloader_iterator = iter(train_loader)

 images, labels = dataloader_iterator.next() // the error throws as soon as this line execute
6

Sorry if I wasn’t clear but it would be helpful to debug if you could provide a small example that I can run which produces the error. It seems that __getitem__ returns something odd and I’m guessing it would be the label but it’s hard to debug without any sample code.

1 Like
7

[Update]
Converting label to int8 solved this issue. Thank you. I am accepting your earlier comments as solution.

Below is the sample getitem for a single index.

getitem tensor([[[0., 0., 0., …, 0., 0., 0.],
[0., 0., 0., …, 0., 0., 0.],
[0., 0., 0., …, 0., 0., 0.],
…,
[0., 0., 0., …, 0., 0., 0.],
[0., 0., 0., …, 0., 0., 0.],
[0., 0., 0., …, 0., 0., 0.]]], dtype=torch.float64) label [‘1’]

One more thing I would like to add. Here, batch_size is 16. I kept the printing for each index. It seems that, get_item is called 16 times (prints all 16 tensors), then the issue occurs. Probably, when it wants to access the second batch, it can not do that and gives the error.