ValueError: too many values to unpack (expected 2), TrainLoader is showing wierd error

 class TrainLoader(Dataset):
 
   def __init__(self,im_dir,ann_dir,img_idx,transform=None):
      self.im_dir=im_dir
      self.ann_dir=ann_dir
      self.img_idx=img_idx
      self.transform=transform
    
  def __len__(self):
     return len(self.img_idx)

  def __getitem__(self,idx):
      print('read')
      image=plt.imread(glob.glob(self.im_dir+ '*_%08d.jpg'%int(self.img_idx[idx]))[0])
      image=(image-np.mean(image))/np.std(image)
      lb=plt.imread(glob.glob(self.ann_dir+ '*_%08d.png'%int(self.img_idx[idx]))[0])*255
      image=self.transform(image)
      lb=torch.from_numpy(lb)
      print(self.img_idx[idx])
  
      return image,lb
    
 transform=transforms.Compose([transforms.ToTensor()])
 t_images,t_label=TrainLoader(im_dir=train_im_dir,ann_dir=train_ann_dir,img_idx=t_lb,transform=transform)

read
00017104
read
00016549
read
00002785

ValueError Traceback (most recent call last)
in ()
1 transform=transforms.Compose([transforms.ToTensor()])
----> 2 t_images,t_label=TrainLoader(im_dir=train_im_dir,ann_dir=train_ann_dir,img_idx=t_lb,transform=transform)
3
4

ValueError: too many values to unpack (expected 2)

I’m working on ADE20k dataset ,Can someone help me with this?

@rasbt @vmirly1 @ptrblck please help me with this, i’m stuck on it for quite some time.

It looks like you are trying to get the first batch from the initialization of your DataLoader.
Could you try to first instantiate your DataLoader, then get the batches in a for loop:

train_loader = TrainLoader(im_dir=...)
for t_images, t_label in train_loader:
    print(t_images.shape)
2 Likes

The answer from @ptrblck should solve the problem. But to add more, the problem is in the following line:

Here TrainLoader is a class, and you want to create an instance (object) from this class. I would change the name to TrainSet t avoid confusion with a loader. So, on the left-hand side there should be only one variable as an object created from the class TrainSet (not a tuple):

train_set = TrainSet(im_dir=train_im_dir,ann_dir=train_ann_dir,img_idx=t_lb,transform=transform)

Then, once you have created an instance of TrainSet, then should be able to use that in a for-loop to get samples.

Note that while the above works in an infinite loop that gives samples of size 1, you may also need to use torch.utils.data.DataLoader, to get batches of data instead of one sample at a time:

train_set = TrainSet(im_dir=train_im_dir,ann_dir=train_ann_dir,img_idx=t_lb,transform=transform)
train_loader = utils.data.DataLoader(dataset=train_set,
                                     batch_size=32,
                                     shuffle=True,
                                     num_workers=1) 

Then, you can get batches of data in a for loop:

for batch_x, batch_y in train_loader:
    print(batch_x.shape, batch_y.shape)
2 Likes

How it is running for first three iteration but not stopping on rather first iteration?

Do you mean you want to iterate 3 times? If that’s true, you can do that using enumerate() and break the loop after 3 iterations as follows:

for i,(batch_x, batch_y) in enumerate(train_loader):
     print(batch_shape, batch_y.shape)
    if i == 2:
        break

Alternatively, you can do it as follows:

for i in range(3):
     batch_x, batch_y = next(iter(train_loader))
     print(batch_x,shape, batch_y.shape)