Concat image datasets with different size and number of channels

#1

Good evening everyone,

I got a problem with loading multiple datasets and did not find a solution so far.
For loading multiple datasets into one dataloader I can use the ConcatDataset class, but how do I concatenate e.g. CIFAR10 and MNIST. For this I would have CIFAR10 to resize and convert to grayscale without applying the transformations to MNIST.

Any ideas on how to do that?

1 Like
How does ConcatDataset work?
#2

You could specify the transformations for each dataset.
This code should work:

mnist = datasets.MNIST(
    root='/home/ptrblck/python/data',
    download=False,
    transform=transforms.Compose([
        transforms.ToTensor()
    ])
)

cifar = datasets.CIFAR10(
    root='/home/ptrblck/python/data',
    download=False,
    transform=transforms.Compose([
        transforms.Grayscale(),
        transforms.Resize(28),
        transforms.ToTensor()
    ])
)

dataset = ConcatDataset((mnist, cifar))

As you can see transforms.Resize and transforms.Grayscale was only applied for the CIFAR10 dataset.

1 Like
#3

Oh yes of course, I am really a blockhead. :grin:
Thanks!

#4

Can you concatenate the targets using this approach? If not, how do we go about it?

#5

ConcatDataset will concatenate all passed Datasets keeping their samples and targets.

How does ConcatDataset work?
#6

may I ask what this means? I’d lake to take the union for the labels and being relabeled from from scratch from 0 to len_dataset1+len_dataset2.

easiest solution to what I want is to do use this: Does Concatenate Datasets preserve class labels and indices - #12 by Brando_Miranda by using learn2learn’s union of data sets.

#7

ConcatDataset will just iterate both datasets as seen here.
Here is a small example:

dataset1 = TensorDataset(torch.arange(5).float().view(-1, 1))
dataset2 = TensorDataset(torch.arange(5).float().view(-1, 1) + 0.1)

dataset = torch.utils.data.ConcatDataset((dataset1, dataset2))

for d in dataset:
    print(d)

# (tensor([0.]),)
# (tensor([1.]),)
# (tensor([2.]),)
# (tensor([3.]),)
# (tensor([4.]),)
# (tensor([0.1000]),)
# (tensor([1.1000]),)
# (tensor([2.1000]),)
# (tensor([3.1000]),)
# (tensor([4.1000]),)

loader = DataLoader(dataset, batch_size=2)
for d in loader:
    print(d)
    
# [tensor([[0.],
#         [1.]])]
# [tensor([[2.],
#         [3.]])]
# [tensor([[4.0000],
#         [0.1000]])]
# [tensor([[1.1000],
#         [2.1000]])]
# [tensor([[3.1000],
#         [4.1000]])]

loader = DataLoader(dataset, batch_size=2, shuffle=True)
for d in loader:
    print(d)

# [tensor([[0.1000],
#         [1.1000]])]
# [tensor([[3.1000],
#         [2.0000]])]
# [tensor([[1.],
#         [0.]])]
# [tensor([[4.],
#         [3.]])]
# [tensor([[4.1000],
#         [2.1000]])]

Is this what you are referring to as “union” or are samples duplicated in both datasets which you want to remove?

#8

no, I just mean concatenate and re-index the class. Basically assumes mutually exclusive labels.

#9

In that case my code should work and shows this behavior.

#10

but you still need to produce the right labels. It’s not hard I suppose but if the data loader is random we want the classes to be consistently & correct labeled.

#11

The datasets are indexed separately as seen in the linked code, so they cannot be mixed up.
You can use my code snippet to add targets and would see that the correspondence is not broken (it would otherwise make this class quite useless):

dataset1 = TensorDataset(torch.arange(5).float().view(-1, 1), torch.zeros(5, 1))
dataset2 = TensorDataset(torch.arange(5).float().view(-1, 1) + 0.1, torch.ones(5, 1))

dataset = torch.utils.data.ConcatDataset((dataset1, dataset2))

loader = DataLoader(dataset, batch_size=2, shuffle=True)
for d in loader:
    print(d)