# A runtime error occurred in creating and using custom dataset

**URL:** <https://discuss.pytorch.org/t/a-runtime-error-occurred-in-creating-and-using-custom-dataset/14091>\
**Category:** Uncategorized\
**Created:** [February 27, 2018, 10:57am UTC](https://discuss.pytorch.org/t/a-runtime-error-occurred-in-creating-and-using-custom-dataset/14091 "2018-02-27T10:57:39Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![minhoha](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/minhoha/32/2859_2.png) [@minhoha](https://discuss.pytorch.org/u/minhoha)\
**Post date:** [February 27, 2018, 10:57am UTC](https://discuss.pytorch.org/t/a-runtime-error-occurred-in-creating-and-using-custom-dataset/14091/1 "2018-02-27T10:57:39Z")

</div>

I created a custom dataset with reference to github.  
([GitHub - utkuozbulak/pytorch-custom-dataset-examples: Some custom dataset examples for PyTorch](https://github.com/utkuozbulak/pytorch-custom-dataset-examples#future-updates))  
The dataset is a variant of cifar100 and consists of .png images and .csv file with the picture directory and label.

The dataset code is as follows.

> from torch.utils.data.dataset import Dataset  
> from torchvision import transforms  
> import pandas as pd  
> import numpy as np  
> from PIL import Image
> 
> class CIFAR100DIRTY\_TEST(Dataset):
> 
> ```
> def __init__ (self, csv_path):
> self.transformations = transforms.Compose([transforms.CenterCrop(32),
> transforms.ToTensor(),
> transforms.Normalize(mean=[0.485, 0.456, 0.406],std=[0.229, 0.224, 0.225])])
> self.data_info = pd.read_csv(csv_path, header=None)
> self.image_arr = np.asarray(self.data_info.iloc[:, 0])
> self.label_arr = np.asarray(self.data_info.iloc[:, 1])
> self.data_len = len(self.data_info.index)
> 
> def __getitem__ (self, index): # returns the data and labels. This function is called from dataloader like this
> single_image_name = self.image_arr[index]
> img_as_img = Image.open(single_image_name)
> img_as_tensor = self.transformations(img_as_img)
> single_image_label = self.label_arr[index]
> 
> return (img_as_tensor, single_image_label)
> 
> def __len__ (self):
> return self.data_len
> 
> ```

And in the main file I called:

> cifar\_train\_dirty = cifar\_dirty\_train.CIFAR100DIRTY\_TRAIN(“/home/mhha/A2S/cifar100\_train\_targets.csv”)  
> cifar\_test\_dirty = cifar\_dirty\_test.CIFAR100DIRTY\_TEST(“/home/mhha/A2S/cifar100\_test\_targets.csv”)
> 
> train\_loader = torch.utils.data.DataLoader(cifar\_train\_dirty,batch\_size=args.bs, shuffle=True,num\_workers=2,drop\_last=False)  
> test\_loader = torch.utils.data.DataLoader(cifar\_test\_dirty,batch\_size=10000, shuffle=False,num\_workers=2,drop\_last=False)

```auto
Traceback (most recent call last):
  File "main_20180227_thin_v1.py", line 144, in <module>
    train(epoch)
  File "main_20180227_thin_v1.py", line 88, in train
    for batch_idx, (inputs, targets) in enumerate(train_loader):
  File "/home/mhha/.conda/envs/pytorchmh2/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 200, in __next__
    return self._process_next_batch(batch)
  File "/home/mhha/.conda/envs/pytorchmh2/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 220, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
  File "/home/mhha/.conda/envs/pytorchmh2/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 40, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/home/mhha/.conda/envs/pytorchmh2/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 109, in default_collate
    return [default_collate(samples) for samples in transposed]
  File "/home/mhha/.conda/envs/pytorchmh2/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 109, in <listcomp>
    return [default_collate(samples) for samples in transposed]
  File "/home/mhha/.conda/envs/pytorchmh2/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 100, in default_collate
    return torch.LongTensor(batch)
RuntimeError: tried to construct a tensor from a int sequence, but found an item of type numpy.int64 at index (1)

```

Which part is wrong? How is it better to modify it?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [February 27, 2018, 3:42pm UTC](https://discuss.pytorch.org/t/a-runtime-error-occurred-in-creating-and-using-custom-dataset/14091/2 "2018-02-27T15:42:10Z")

</div>

It seems `single_image_label` is a numpy array. Try to return the labels as `Tensors`:

```
single_image_label = torch.from_numpy(self.label_arr[index])
```

---

<div class="post-metadata">

**Author:** ![minhoha](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/minhoha/32/2859_2.png) [@minhoha](https://discuss.pytorch.org/u/minhoha)\
**Post date:** [February 28, 2018, 6:26am UTC](https://discuss.pytorch.org/t/a-runtime-error-occurred-in-creating-and-using-custom-dataset/14091/3 "2018-02-28T06:26:17Z")

</div>

Unfortunately, it doesn’t work…

---

<div class="post-metadata">

**Author:** ![jpeg729](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jpeg729/32/3464_2.png) [@jpeg729](https://discuss.pytorch.org/u/jpeg729)\
**Post date:** [February 28, 2018, 7:22am UTC](https://discuss.pytorch.org/t/a-runtime-error-occurred-in-creating-and-using-custom-dataset/14091/4 "2018-02-28T07:22:33Z")

</div>

Does it still produce the same error?

---

<div class="post-metadata">

**Author:** ![minhoha](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/minhoha/32/2859_2.png) [@minhoha](https://discuss.pytorch.org/u/minhoha)\
**Post date:** [February 28, 2018, 7:24am UTC](https://discuss.pytorch.org/t/a-runtime-error-occurred-in-creating-and-using-custom-dataset/14091/5 "2018-02-28T07:24:03Z")

</div>

I fixed it.

single\_image\_label = np.int(self.label\_arr[index])
