I have two dataloaders which have different sizes and i want to enumerate over them so that I can fetch one batch from each alternately. The code below shows how I implement this and then fetch the data from the dataloader.
alexnet_train_loader is the shorter one. (returns input image and target)
siamese_train_loader is the longer one. (returns two images)
#create the combined loader
alex_extended = itertools.cycle(alexnet_train_loader)
zipped_loader = itertools.zip_longest(alex_extended, siamese_train_loader, fillvalue=None)
combined_loader = itertools.takewhile(lambda t: t[1] != None, zipped_loader)
for i, data in enumerate(combined_loader):
#siamesenet part
input1 = data[1][0]
input2 = data[1][1]
#alexnet part
input = data[0][0]
target = data[0][1]
I create the combined_loader freshly for each epoch through my longer siamese_train_loader data as i saw that once itertools objects are consumed, there’s no way to seek to the beginning.
Is this the correct way or do you suggest any other way which is simpler?