I have two dataloaders which have different sizes and i want to enumerate over them so that I can fetch one batch from each alternately. The code below shows how I implement this and then fetch the data from the dataloader.
alexnet_train_loader is the shorter one. (returns input image and target)
siamese_train_loader is the longer one. (returns two images)
#create the combined loader alex_extended = itertools.cycle(alexnet_train_loader) zipped_loader = itertools.zip_longest(alex_extended, siamese_train_loader, fillvalue=None) combined_loader = itertools.takewhile(lambda t: t != None, zipped_loader) for i, data in enumerate(combined_loader): #siamesenet part input1 = data input2 = data #alexnet part input = data target = data
I create the combined_loader freshly for each epoch through my longer siamese_train_loader data as i saw that once itertools objects are consumed, there’s no way to seek to the beginning.
Is this the correct way or do you suggest any other way which is simpler?