Why my dataloader swaps the batch size to the 2nd dimension automatically

data
1

I am preparing data for an RNN, the data point in the dataset is a list containing 11 RGB images, thus the dimension of “out” in the following is 11 * 3 * 224 * 224:

def getitem(self,index):

    topk_s = self.tops[index]
    if self.shuffle:
        topk_s = random.sample(topk_s,len(topk_s))

    out = []
    
    for i in topk_s:
        img = Image.open(os.path.join(self.slides, self.grid[i])).convert('RGB')
        if self.mult != 1:
            img = img.resize((224,224), Image.BILINEAR)
        if self.transform is not None:
            img = self.transform(img)
        out.append(img)
    
    return out

the dimension of dataset is:
(614, 11, 3, 224, 224)

But after making a dataloader, the batch size (128) is swapped to 2nd dimension, why would this happen?

image
image491×515 48.8 KB

Thank you!

2

I cannot reproduce the issue and get the expected shapes:

class MyDataset(Dataset):
    def __init__(self):
        pass
    
    def __len__(self):
        return 614
    
    def __getitem__(self, index):
        out = torch.randn(11, 3, 224, 224)
        return out
    
dataset = MyDataset()
x = dataset[0]
print(x.shape)
# torch.Size([11, 3, 224, 224])

loader = DataLoader(dataset, batch_size=128)
for data in loader:
    print(data.shape)
# torch.Size([128, 11, 3, 224, 224])
# torch.Size([128, 11, 3, 224, 224])
# torch.Size([128, 11, 3, 224, 224])
# torch.Size([128, 11, 3, 224, 224])
# torch.Size([102, 11, 3, 224, 224])
1 Like
4

Thank you! I learned a lot from your answer to simplify a tricky question!

class MyDataset(data.Dataset):

def __init__(self):
    pass

def __len__(self):
    return 614

def __getitem__(self, index):
    out = []
    for i in range(11):
        out.append(torch.randn(3, 224, 224))
    return out

dataset = MyDataset()
x = dataset[0]
print(len(x),len(x[0]),len(x[0][0]),len(x[0][0][0]))

loader = torch.utils.data.DataLoader(dataset, batch_size=128)
for i, data1 in enumerate(loader):
print(len(data1),len(data1[0]),len(data1[0][0]),len(data1[0][0][0]),len(data1[0][0][0][0]))

The answer is:
11 3 224 224
11 128 3 224 224
11 128 3 224 224
11 128 3 224 224
11 128 3 224 224
11 102 3 224 224

But I am still not sure if DataLoader swapped the 2 dimensions automatically for convenience of users to train RNN.

5

If you are returning a list from the __getitem__ PyTorch will keep it and x as well as data1 will both be lists containing the tensors. If you out = torch.stack(out) the list in __getitem__ the same output shapes seem in my code snippet will be returned.

1 Like
6

You are right. Thank you!

I have confirmed that data1 gives lists with swapped dimensions.