One Question about 60-min blitz sys.meta_path is None, Python is likely shutting down

1

import torch
import torchvision
import torchvision.transforms as transforms
import matplotlib.pyplot as plt
import numpy as np

transform = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root=’./data’, train=True,
download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,
shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root=’./data’, train=False,
download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=4,
shuffle=False, num_workers=2)

classes = (‘plane’, ‘car’, ‘bird’, ‘cat’,
‘deer’, ‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’)

functions to show an image

def imshow(img):
img = img / 2 + 0.5 # unnormalize
npimg = img.numpy()
plt.imshow(np.transpose(npimg, (1, 2, 0)))
#plt.show()

get some random training images

dataiter = iter(trainloader)
images, labels = dataiter.next()

show images

imshow(torchvision.utils.make_grid(images))
plt.show()

print labels

print(’ ‘.join(’%5s’ % classes[labels[j]] for j in range(4)))

The pictures can’t be shown in my computer complied by Pycharm. I use the Pycharm to connect the Server(CentOS).
But the pictures can be shown in my computer compiled by JupyterNotebook. I am very confused .Please Help me!
Error Message:
ssh://cong_pan@172.18.11.12:22/data/cong_pan/anaconda3/bin/python3.6 -u /data/cong_pan/jiaqi_wang/code/pytorch/showphoto.py
Files already downloaded and verified
Files already downloaded and verified
dog horse car horse
Exception ignored in: <bound method DataLoaderIter.del of <torch.utils.data.dataloader.DataLoaderIter object at 0x7f7253d55eb8>>
Traceback (most recent call last):
File “/data/cong_pan/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 333, in del
File “/data/cong_pan/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 319, in _shutdown_workers
File “/data/cong_pan/anaconda3/lib/python3.6/multiprocessing/queues.py”, line 345, in get
ImportError: sys.meta_path is None, Python is likely shutting down

Process finished with exit code 0

2

Similar issue when run the tutorial.
The code is exactly the same code as shown in the tutorial.

import torch
import torchvision
import torchvision.transforms as transforms

transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,
                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,
                                       download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=4,
                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')



import matplotlib.pyplot as plt
import numpy as np

# functions to show an image

def imshow(img):
    img = img / 2 + 0.5     # unnormalize
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))


# get some random training images
dataiter = iter(trainloader)
images, labels = dataiter.next()

# show images
imshow(torchvision.utils.make_grid(images))
# print labels
print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

Error message

Files already downloaded and verified
Files already downloaded and verified
plane  deer truck   cat
Exception ignored in: <bound method DataLoaderIter.__del__ of <torch.utils.data.dataloader.DataLoaderIter object at 0x7f8850688518>>
Traceback (most recent call last):
  File "/data/apollo/a/Xiaoke/anaconda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 333, in __del__
  File "/data/apollo/a/Xiaoke/anaconda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 319, in _shutdown_workers
  File "/data/apollo/a/Xiaoke/anaconda/lib/python3.6/multiprocessing/queues.py", line 337, in get
ImportError: sys.meta_path is None, Python is likely shutting down

System info:
ubuntu 14.04
python3.6
anconda
torch gpu
I am ssh the server…

Thanks a lot if anyone can give the help.

3

Experiencing the same issue as well

macos 10.13.4
python 3.6.5
torch 0.3.1
torchvision 0.2.0

4

Hi everyone! Thanks for reporting. I am aware of the issue and will have a fix for the next release. The error you saw was harmless and can be ignored. :slight_smile:

1 Like
5

I solve the problem by removing “dataiter = iter(trainloader)”.Instead, I use the “iter” directly. Like that:
“images, labels = iter(trainloader).next()”

6

this is extremely troublesome if you have shuffle=False…

7

I have test it on my machine. It’s Okay.
There is no problem, even I change shuffle=False.

8

I see it as @SimonW.
It won’t cause an error, but you will most likely get the same batch over and over again.
Have a look at this small example:


class MyDataset(Dataset):
    def __init__(self):
        self.data = torch.arange(100)
        
    def __getitem__(self, index):
        return self.data[index]
    
    def __len__(self):
        return len(self.data)

dataset = MyDataset()
loader = DataLoader(dataset,
                    batch_size=1,
                    shuffle=False)

data = iter(loader).next()
print(data)
data = iter(loader).next()
print(data)
# Prints 0s every time

data_iter = iter(loader)
data = data_iter.next()
print(data)
data = data_iter.next()
print(data)
# Gets new batch (0, 1, ...)
9

It won’t error. But you will get the same batch every time.

10

I have the same error, also using pycharm and no images are displayed.

Any help would be apreciated,

Thanks.

11

I solved the iterator issue by calling the following before the program exits:

data_iter._shutdown_workers()

I solved the display issue by calling the following immediately after plt.imshow:

plt.show()