DataLoader for Train + Val Split

1

Hi, I’m trying to figure out a neat way of using dataloaders for train and val.

train_samples = DataLoader("TRAIN")
val_samples = DataLoader("VAL")

for epoch in range(n_epochs):
      for iteration, aSample in enumerate(train_samples):
          train(aSample)

What’s a neat way to incorporate val_samples in every iteration?

next(val_samples)

will not work because the DataLoader is not an iterator…

I can make val_samples a generator, but I’ll need to restart the generator every time it runs out, and restarting it takes a minute or so. (Num_workers, reading data split from disk etc.) Also multiple restarts per epoch since val samples are fewer than train.

2

Use a custom dataset.