ImportError: dlopen: cannot load any more object with static TLS (torch and cv2)

1

Hi, I’m trying to import both pytorch and opencv with in python 2.7, but I faced with the following error.

(pytorch) root@hzxs-ai-deep-26:/mnt/storage01/xuyi/videoTag # python
Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul  2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import torch
>>> import cv2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen: cannot load any more object with static TLS
>>> torch.__version__
'0.2.0_4'
>>> exit()

or

(pytorch) root@hzxs-ai-deep-26:/mnt/storage01/xuyi/videoTag # python
Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul  2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import cv2
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/anaconda3/envs/pytorch/lib/python2.7/site-packages/torch/__init__.py", line 53, in <module>
    from torch._C import *
ImportError: dlopen: cannot load any more object with static TLS

No matter which order to import, it happens always.

my python version is 2.7.12,
torch version is 0.2.0_4
opencv with ffmpeg is 3.3.1
OS is debian 4.9.0-0.bpo.3-amd64.

Thanks a lot for any suggestion.

2

I have found one solution but not perfect for this kind of problem. I hope the next version of pytorch can fix such problem.

We write a script called “main.py”, in which we use the pytorch but without opencv. And as for opencv operation, we are mainly aimed to process data, so I create another script called “datautils.py”, in which we import opencv without using any operation in pytorch.

In order to get the data from datautils.py, we can take measures as following. For example
in main.py

import torch
# some operation in torch
from subprocess import PIPE,Popen
import pickle
p = Popen('python datautils.py ' + argv, shell=True, stdout=PIPE, stderr=PIPE)
out,err = p.communicate()
data = pickle.loads(out)
# some operation in torch

In datautils.py

import cv2
import pickle
if __name__ == "__main__":
    cap = cv2.VideoCapture(argv[1])
    has_next, img = cap.read()
    # and some other operation in cv2
    print(pickle.dumps(data))

It works in debian.