I want to perform certain post-processing on each of the MNIST digits but I get
OSError: [Errno 24] Too many open files. I am using
pytorch==0.4.1. Have tried this both for python
2.7 and 3.6.
I have also tried the other suggestions mentioned in this i.e I have rebooted my ubuntu server as well as tried setting ulimit but I get
ulimit: value exceeds hard limit when I try to set something above 4096. Although, I get the output as
unlimited when I run
ulimit. Is there any other solution for this?.
My code to get the train and the test loaders:
import torch from torch.utils.data import DataLoader from torchvision.utils import save_image import torchvision.datasets as dset import torchvision.transforms as transforms trans = transforms.ToTensor() train_set = dset.MNIST(root=root, train=True, transform=trans, download=True) test_set = dset.MNIST(root=root, train=False, transform=trans) train_loader = DataLoader(dataset=train_set, batch_size=1, shuffle=True, num_workers=4) test_loader = DataLoader(dataset=test_set, batch_size=1, shuffle=True, num_workers=4)