How to read a dataset in .mat form in pytorch

This post might be helpful, if you would like to read these .mat files lazily.
However, it seems your dataset might be stored completely in the file.
In that case you could read it outside of the Dataset using scipy.io, transform them to tensors via torch.from_numpy and use a TensorDataset.