How to iterate through VCTK dataset?


When I worked with CIFAR dataset. I followed this instructions to go through dataset:

    for batch_idx, batch in enumerate(training_data_loader):
        input = Variable(batch[0])

Now I want to work with audio and I use VCTK dataset.

train_set = datasets.VCTK(root = 'processed/', download = True)
training_data_loader = DataLoader(dataset = train_set,
            num_workers=opt.nThreads, batch_size=opt.batchSize,shuffle=True)

How can I do same thing with this dataset?

It shows error after writing this:

for batch_idx, batch in enumerate(training_data_loader):
    print batch_idx
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 37625 and 44415 in dimension 1 at /opt/conda/conda-bld/pytorch_1518238409320/work/torch/lib/TH/generic/THTensorMath.c:2897

The length of the elements in VCTK are not the same. You need to pad/clip each sample either in a transformation which you add to train_set or a custom collate_fn in training_data_loader. The first option will will make all your samples the same length. The second option allows for the possibility of cropping/clipping each batch individually (to the length of the longest sample for example).

1 Like