I have a hdf5 file that contains picture of a certain number of people, from a certain number of source cameras, for many seconds. So it is like this: file[seconds][person][camera] But this is quite irregular, such that for a given second, there may be different number of persons, and for a given second and person there may be picture from different cameras. I want to create a map-style pytorch dataset, so I need to implement get_item(idx) that will return a unique second, person and camera for that idx.
My first idea is to iterate through the whole dataset and create dictionaries that can be accessed with idx, that is, second[idx] = this_second, person[idx] = this_person, camera[idx] = this_camera. So I can use all of that to get a unique data from the dataset with file[this_second][this_person][this_camera].
However this solution seems to complicated for me. I wonder if there is a better way to solve that, since this is probably a common problem.