Use two imagefolders in dataset

I am trying to create a dataset with an imagefolder as input and another imagefolder to use as labels. Is there any way I can have them both in the same DataLoader?

My code is as follows:

dataset = torchvision.datasets.ImageFolder(root=TRAIN_DATA_PATH, transform=TRANSFORM_IMG)
dataset_labels = torchvision.datasets.ImageFolder(root=TRAIN_LABELS_PATH, transform=TRANSFORM_IMG)

I would like to be able to integrate dataset_labels into dataset, so that I can then just create a DataLoader from dataset.

use torch.utils.data.ConcatDataset