Broken torchtext conda package?

1

I need to use the torchtext package to run a certain model, and I have been having a strange bug that trying to run ‘import torch’ at any point causes the process to hang indefintely. I pinned pytorch to 1.X because otherwise it can only choose torchtext=0.6.0 for pytorch=2.X, and I need a newer version. I also posted a workaround at the bottom of the post for those who need it.

I stripped it down to the following conda environment and if I execute this:

conda create -n test pytorch=1 python=3.11 torchtext=0.14 -c nvidia -c pytorch -c defaults -c conda-forge
conda activate test
python -c "import torch"

it hangs indefinitely. If I run
python -v -c "import torch"

to see what is going on, I see that the final lines where it hangs are the following import:

import 'torch._utils_internal' # <_frozen_importlib_external.SourceFileLoader object at 0x7c0442d3a050>
# /home/schoonho/anaconda3/envs/test/lib/python3.11/site-packages/torch/__pycache__/torch_version.cpython-311.pyc matches /home/schoonho/anaconda3/envs/test/lib/python3.11/site-packages/torch/torch_version.py
# code object from '/home/schoonho/anaconda3/envs/test/lib/python3.11/site-packages/torch/__pycache__/torch_version.cpython-311.pyc'
# /home/schoonho/anaconda3/envs/test/lib/python3.11/site-packages/torch/__pycache__/version.cpython-311.pyc matches /home/schoonho/anaconda3/envs/test/lib/python3.11/site-packages/torch/version.py
# code object from '/home/schoonho/anaconda3/envs/test/lib/python3.11/site-packages/torch/__pycache__/version.cpython-311.pyc'
import 'torch.version' # <_frozen_importlib_external.SourceFileLoader object at 0x7c0442dab250>
import 'torch.torch_version' # <_frozen_importlib_external.SourceFileLoader object at 0x7c0442d3a390>
# /home/schoonho/anaconda3/envs/test/lib/python3.11/site-packages/torch/__pycache__/_six.cpython-311.pyc matches /home/schoonho/anaconda3/envs/test/lib/python3.11/site-packages/torch/_six.py
# code object from '/home/schoonho/anaconda3/envs/test/lib/python3.11/site-packages/torch/__pycache__/_six.cpython-311.pyc'
import 'torch._six' # <_frozen_importlib_external.SourceFileLoader object at 0x7c0442dab590>

Fix:
Weirdly, if I change the Python version to 3.10 in the conda environment it does work:

conda create -n test pytorch=1 python=3.10 torchtext=0.14 -c nvidia -c pytorch -c defaults -c conda-forge

To conclude, I can just continue using python 3.10 and all will be fine, but it is strange that this happens I think.

Not sure if I am doing something wrong but it seems that if conda finds this a valid environment then maybe something is wrong in the conda packaging.

Information on broken environment
Ubuntu 22.04.4 LTS
Nvidia driver 555.42.06
pytorch 1.13.1 (cuda112py311)
torchtext 0.14.1
cudatoolkit 11.7.0

I can post the full environment if desirable.

2

So I have a suspicion of what is going on. For the python 3.10 env it installs:

pytorch      1.13.1      py3.10_cuda11.7_cudnn8.5.0_0    pytorch

and for the python 3.11 env it installs:

pytorch      1.13.1      cuda112py311h13fee9e_200    conda-forge

So probably its a broken conda-forge pytorch package that is not playing nicely with torchtext.

3

Yes, this could be the reason as we do not maintain or build the conda-forge PyTorch binaries. The “hang” sounds a bit as if PyTorch wasn’t compiled for your GPU architecture and CUDA JIT might try to compile it live.

1 Like
4

I did not understand what could be causing the hang but this seems like a good candidate. Thanks for the response!