ImportError after upgrading pytorch and torchvision inside my conda environment

Hi!

I recently have to upgrade the version of pytorch from 1.1 to 1.3.1. However, I’m getting this error :frowning:

ImportError: /miniconda3/envs/base/lib/python3.7/site-packages/torchvision/_C.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN3c1011CPUTensorIdEv

Any clue how to fix it?

BTW, I don’t plan to use FasterRCNN, so a dirty hack to disable the conflicting stuff is also welcome :grin: .

Thanks in advance!
Victor


Conda environment after the update

conda list
# packages in environment at /home/SERILOCAL/v.castillo/install/bin/miniconda3/envs/tsm-saic-dev:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
backcall                  0.1.0                    pypi_0    pypi
blas                      1.0                         mkl  
bzip2                     1.0.8                h7b6447c_0  
ca-certificates           2020.7.22                     0  
cairo                     1.14.12              h8948797_3  
certifi                   2020.6.20                py37_0  
cffi                      1.13.2           py37h2e261b9_0  
cudatoolkit               10.0.130                      0  
cudnn                     7.6.5                cuda10.0_0  
decorator                 4.4.2                      py_0  
docopt                    0.6.2                      py_1    conda-forge
easydict                  1.9                      pypi_0    pypi
epic-kitchens             1.7.1                    pypi_0    pypi
ffmpeg                    4.0                  hcdf2ecd_0  
fontconfig                2.13.0               h9420a91_0  
freeglut                  3.0.0                hf484d3e_5  
freetype                  2.9.1                h8a8886c_1  
glib                      2.63.1               h5a9c865_0  
graphite2                 1.3.13               h23475e2_0  
gulpio                    540.66                     py_0    conda-forge
h5py                      2.8.0            py37h989c5e5_3  
harfbuzz                  1.8.8                hffaf4a1_0  
hdf5                      1.10.2               hba1933b_1  
icu                       58.2                 h9c2bf20_1  
intel-openmp              2020.0                      166  
ipdb                      0.13.2                   pypi_0    pypi
ipykernel                 5.3.4            py37h5ca1d4c_0  
ipython                   7.13.0                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipython_genutils          0.2.0                    py37_0  
jasper                    2.0.14               h07fcdf6_1  
jedi                      0.16.0                   pypi_0    pypi
jinja2                    2.11.1                     py_0    conda-forge
joblib                    0.14.1                     py_0  
jpeg                      9b                   h024ee3a_2  
jupyter_client            6.1.6                      py_0  
jupyter_core              4.6.3                    py37_0  
ld_impl_linux-64          2.33.1               h53a641e_7  
libedit                   3.1.20181209         hc058e9b_0  
libffi                    3.2.1                hd88cf55_4  
libgcc-ng                 9.1.0                hdf63c60_0  
libgfortran-ng            7.3.0                hdf63c60_0  
libglu                    9.0.0                hf484d3e_1  
libopencv                 3.4.2                hb342d67_1  
libopus                   1.3                  h7b6447c_0  
libpng                    1.6.37               hbc83047_0  
libsodium                 1.0.18               h7b6447c_0  
libstdcxx-ng              9.1.0                hdf63c60_0  
libtiff                   4.1.0                h2733197_0  
libuuid                   1.0.3                h1bed415_2  
libvpx                    1.7.0                h439df22_0  
libxcb                    1.13                 h1bed415_1  
libxml2                   2.9.9                hea5a465_1  
markupsafe                1.1.1            py37h8f50634_1    conda-forge
mkl                       2020.0                      166  
mkl-service               2.3.0            py37he904b0f_0  
mkl_fft                   1.0.15           py37ha843d7b_0  
mkl_random                1.1.0            py37hd6b4f25_0  
ncurses                   6.1                  he6710b0_1  
ninja                     1.9.0            py37hfd86e86_0  
numpy                     1.18.1           py37h4f9e942_0  
numpy-base                1.18.1           py37hde5b4d6_1  
olefile                   0.46                     py37_0  
opencv                    3.4.2            py37h6fd60c2_1  
opencv-python             4.2.0.34                 pypi_0    pypi
openssl                   1.1.1g               h7b6447c_0  
pandas                    1.0.3                    pypi_0    pypi
parso                     0.6.2                    pypi_0    pypi
pcre                      8.43                 he6710b0_0  
pexpect                   4.8.0                    py37_1  
pickleshare               0.7.5                 py37_1001  
pillow                    6.2.0            py37h34e0f95_0  
pip                       20.0.2                   py37_1  
pixman                    0.38.0               h7b6447c_0  
prompt-toolkit            3.0.5                      py_0  
protobuf                  3.11.3                   pypi_0    pypi
ptyprocess                0.6.0                    pypi_0    pypi
py-opencv                 3.4.2            py37hb342d67_1  
pycparser                 2.19                     py37_0  
pygments                  2.6.1                      py_0  
python                    3.7.6                h0371630_2  
python-dateutil           2.8.1                      py_0  
python_abi                3.7                     1_cp37m    conda-forge
pytorch                   1.3.1           py3.7_cuda10.0.130_cudnn7.6.3_0    pytorch
pytz                      2019.3                   pypi_0    pypi
pyzmq                     19.0.1           py37he6710b0_1  
readline                  7.0                  h7b6447c_5  
scikit-learn              0.22.1           py37hd81dba3_0  
scipy                     1.4.1            py37h0b6359f_0  
setuptools                45.1.0                   py37_0  
sh                        1.12.14         py37hc8dfbb8_1002    conda-forge
six                       1.14.0                   py37_0  
sqlite                    3.31.1               h7b6447c_0  
tensorboardx              2.0                      pypi_0    pypi
tk                        8.6.8                hbc83047_0  
torchvision               0.3.0           py37_cu10.0.130_1    pytorch
tornado                   6.0.4            py37h7b6447c_1  
tqdm                      4.42.0                     py_0  
traitlets                 4.3.3                    py37_0  
wcwidth                   0.1.9                    pypi_0    pypi
wheel                     0.34.2                   py37_0  
xz                        5.2.4                h14c3975_4  
zeromq                    4.3.2                he6710b0_2  
zlib                      1.2.11               h7b6447c_3  
zstd                      1.3.7                h0b5b093_0

The output of the most recent conda revision

2020-09-16 08:59:58  (rev 2)
     pytorch  {1.1.0 (defaults/linux-64) -> 1.3.1 (pytorch/linux-64)}
     torchvision  {0.3.0 (defaults/linux-64) -> 0.3.0 (pytorch/linux-64)}

:rofl: updating torchvision got rid of the ImportError

2020-09-16 11:43:27  (rev 3)
     torchvision  {0.3.0 (pytorch/linux-64) -> 0.4.2 (pytorch/linux-64)}

I will keep it unsolved as: (i) maybe something might appear later on; (ii) it would be great to know how to fix/debug this in a principled manner.

The first step usually is to make sure PyTorch and TorchVision versions are compatible. TorchVision has been relatively tightly coupled to PyTorch, at least for most of the year. :slight_smile:

Thanks Tom, good advice and I’m glad that I found a quick fix (almost by luck this time). Yep, it stills puzzles me why it was broken.

Doesn’t conda take care that the versions of pytorch and torchivision (both from the official channel) are in sync?
Asking as I’m missing something regarding the conda recipe used to build the binaries.

I must admit I’m not much of a conda user, but so I think for pip the dependency versioning is very difficult, more so by the mix of release/nightly and cpu/various cuda version/rocm… In Debian/Ubuntu, you’d be able to express this type of dependency, but it seems difficult to do in pip. I don’t know about conda.

Another option might be to warn in TorchVision if the major version of PyTorch is higher than what TorchVision was built with.
Fun fact: A long time ago, installing pip would downgrade PyTorch.

Best regards

Thomas

:+1:t2: maintainig binaries is tough. It’s super easy these days wrt 2014, yet ppl complain about it :joy:

:tophat: tip to pytorch-dev-team behind all the binaries. They did it super well from the early days.