How can I separate data when I use ''torch.utils.data.DataLoader'' loaded the dataset?

1

In my train data, I have two class here.
In following way:

      dataset = dset.ImageFolder(root=opt.dataroot_train,
                                transform=transforms.Compose([
                                    transforms.Resize(opt.imageSize),
                                    transforms.CenterCrop(opt.imageSize),
                                    transforms.ToTensor(),
                                    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
                                ]))

        hazy=[]
        gt=[]
        for gt_i in range(int(len(dataset)/2)):
            gt.append(dataset[gt_i][0]) # changed here
        assert gt
        #####--------------------------------------------------------
        # alternative with bellow
        for hazy_i in range(int(len(dataset)/2), int(len(dataset))):
            hazy.append(dataset[hazy_i][0])
        assert hazy
        dataset=list(zip(gt,hazy))
        print('dataset.size',len(dataset))
        dataloader = torch.utils.data.DataLoader(dataset,batch_size=opt.batSize,
                                                    shuffle=True, num_workers=int(opt.workers))

I can random load these two bounded dataset by data loader. But how can I select gt or hazy from this dataloader?

2

I’m not sure what your use case is.
ImageFolder will create the target based on the subfolders which are in the specified root folder.
You could use these targets to select if the current data samples based on their class.
If you don’t shuffle the DataLoader, you’ll get the samples in the order your dataset contains them.
Could you explain a bit what you are trying to do with these samples?
Maybe there is another (better) solution.

4

I wanna train 2 dataset which are high related with each other. Since I don’t just training every data only time if I read them use 2 dataloaders and don’t shuffle them, the last several data cannot combine as a full batch, that will cause troubles.Once I use shuffle in every dataloader that will make the relation ruined.