Understanding pytorch-data

Hi all,

I am a bit new to pytorch and trying to understand the the data loading module. With reference to the tutorial available here: https://pytorch.org/tutorials/beginner/data_loading_tutorial.html , it can be seen that when the dataset class for this specific example is opening each image in def getitem(self, idx) method and converting to tensor. I want to know when we initialize the object of FaceLandmarksDataset class, do we get all images opened and mapped to dict at once or getitem(self,idx) is implemented when we explicitly call the object with a specific id?

class FaceLandmarksDataset(Dataset):
“”“Face Landmarks dataset.”""

def __init__(self, csv_file, root_dir, transform=None):
        csv_file (string): Path to the csv file with annotations.
        root_dir (string): Directory with all the images.
        transform (callable, optional): Optional transform to be applied
            on a sample.
    self.landmarks_frame = pd.read_csv(csv_file)
    self.root_dir = root_dir
    self.transform = transform

def __len__(self):
    return len(self.landmarks_frame)

def __getitem__(self, idx):
    if torch.is_tensor(idx):
        idx = idx.tolist()

    img_name = os.path.join(self.root_dir,
                            self.landmarks_frame.iloc[idx, 0])
    image = io.imread(img_name)
    landmarks = self.landmarks_frame.iloc[idx, 1:]
    landmarks = np.array([landmarks])
    landmarks = landmarks.astype('float').reshape(-1, 2)
    sample = {'image': image, 'landmarks': landmarks}

    if self.transform:
        sample = self.transform(sample)

    return sample

When you initialize the FaceLandmarkDataset, its __init__ method will be called.
In your example, the csv_file will be loaded and assigned to self.landmarks_frame as well as the root directory and transformation.
Indexing the dataset via dataset[0] or iterating the DataLoader will call into __getitem__ and lazily load each sample.
This makes sure you can work with huge datasets, which wouldn’t fit into your system memory.

Thanks, i understand now