How to stack data in __getitem__ use collatefn?

1

I expand data in getitem, how can I stack it in collatefn?

from torch.utils.data import Dataset
import numpy as np

class BarDataset(Dataset):
    def __init__(self):
        self.data = list(np.random.rand(9))
        pass

    def __getitem__(self, idx):
        # some ops
        out = np.array([self.data[idx] * 2, self.data[idx] * 3])
        return out

    def __len__(self):
        return len(self.data)

train_dataset = BarDataset()

train_loader = torch.utils.data.DataLoader(
    train_dataset,
    batch_size=3,
    pin_memory=True
)

for item in train_loader:
    print(item) 
# len(train_loader) = 3, wants len(train_loader) = 3 * 2
2

If you just want to change the shape of item, you could use item = item.view(6, 1) on it.

3

I want to expand dataset, len(train_loader) = 2 * len(train_loader).
I am sorry that I didn’t say it clearly.

4

Ah OK.
I guess you want to use the same data in your Dataset?
If so, you could just fake a new length and use a modulo operation in index to avoid an out of range error:

class BarDataset(Dataset):
    def __init__(self):
        self.data = list(np.random.rand(9))
        pass

    def __getitem__(self, idx):
        # some ops
        idx = idx % len(self.data)
        out = np.array([self.data[idx] * 2, self.data[idx] * 3])
        return out

    def __len__(self):
        return 2 * len(self.data)
5

Thanks, it works
#####I am padding#####