Load data using the the index in getitem

[Pt_S]

Dear all,

I want to load my data using Dataset which must implement the getitem funciton. The function getitem(self, index) get the data indexed with index. So the data must be linearized.

However, my data is constructed as follows:

1. 64 pictures

2. points of 64 pictures:
   list : [[points of pic1], [points of pic2], … [points of pic64]]
   and [points of pic1] is in the format of [ [p1, p2, p3], [p1, p2, p3, p4, p5], [p1, p2]…[p1, p2, p3]] with different lengths.

Each time I want to get one point and corresponding picture, how can I organize my data and use Dataset index to load the data ?

Regards,
Pt

[InnovArul (Arul)]

You need to write a custom collate function to collect the output of __getitem__() method into a list or tuple. An example of custom collate function:

RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 3 and 2 in dimension 1

Yes. It is not dealing with your exact issue. I was hoping, it might give you some idea. Please try-out the following collate function: import torch import numpy as np import torch.utils.data as data def my_collate(batch): data = torch.stack([item[0].unsqueeze(0) for item in batch], 0) target = torch.Tensor([item[1] for item in batch]) return [data, target] class dataset(data.Dataset): def __init__(self): super(dataset, self).__init__() def __len__(self): …