I really appreciate this tutorial on custom datasets. However, the
torch.utils.data.DataLoader class is only briefly mentioned in it:
However, we are losing a lot of features by using a simple
forloop to iterate over the data. In particular, we are missing out on:
- Batching the data
- Shuffling the data
- Load the data in parallel using
torch.utils.data.DataLoaderis an iterator which provides all these features. Parameters used below should be clear. One parameter of interest is
collate_fn. You can specify how exactly the samples need to be batched using
collate_fn. However, default collate should work fine for most use cases.
Could we possibly get a tutorial on custom dataloaders using the
torch.utils.data.DataLoader class? More specifically, how to interface with its parameters, especially the
collate_fn parameters. Also, an explanation on how to inherit from the abstract base class and a template for the
collate_fn would be nice too.
Again, I really appreciate the effort that goes into the tutorials that are currently available, but I feel that a tutorial on custom dataloaders would answer a lot of questions.