My institution uses one server to store many different datasets while another one to conduct computation. How should we load the data from the data server, such that the loading of data will not become the bottleneck? Does Dataloader have considered this occasion?
Welcome!
Check out the documentation for the torch.utils.data.DataLoader. There is a keyword argument for the number of worker processes to spawn to load and preprocess data. You can use this to try and fix any data loading bottlenecks.