Shuffle in DataLoader

vision
1

I only want to load part of my train set. And I want every part in different epoch is the same. Therefore, I set DataLoader shuffle as False. However, at different epoch these parts are different. My code is here:
get DataLoader:

def get_train_loader(args):

    transform = transforms.Compose([
        transforms.Grayscale(num_output_channels=3),
        transforms.Resize((args.size,args.size)),
        transforms.ToTensor(),
    ])

    if args.dataset == 'CIFAR10':
        dataset = CIFAR10(root=args.data_dir,train=True,
                          transform=transforms.Compose([
                              transforms.Resize(args.size,args.size),
                              transforms.ToTensor(),
                          ]))
    elif args.dataset == 'MNIST':
        dataset = MNIST(root=args.data_dir,train=True,transform=transform)
    else:
        dataset = SmallNORB(root=args.data_dir,train=True,transform=transform)

    print(args.shuffle)
    return DataLoader(
        dataset,
        batch_size=args.batch_size,
        shuffle=args.shuffle,
        num_workers=args.workers,
        pin_memory=True
    )

get data:

for (input_tensor, target) in islice(train_loader,self.data_size):
    #my train code

Is there any way I can get the same part of my train set at my every epoch.

2

You could possibly use the Subset class under torch.utils.data.
If you specify the indices from the original dataset you could create a separate dataloader for this subset. Then, even if you use shuffle in the dataloader, you’d get the same data, just shuffled everytime.
For example, adding from your MNIST dataset code:

dataset = MNIST(root=args.data_dir,train=True,transform=transform)
subset = Subset(dataset, np.arange(0,10))

subset_loader = DataLoader(subset, batch_size=args.batch_size, shuffle=args.shuffle, 
                          num_workers=num_workers, pin_memory=True)
1 Like