Theo
(Theo)
March 16, 2023, 6:36pm
1
I’m running the following code:
!pip install tokenizers
!pip install torchdata
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchtext.datasets import Multi30k
And I get the following error:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchvision 0.14.1+cu116 requires torch==1.13.1, but you have torch 2.0.0 which is incompatible.
torchtext 0.14.1 requires torch==1.13.1, but you have torch 2.0.0 which is incompatible.
torchaudio 0.13.1+cu116 requires torch==1.13.1, but you have torch 2.0.0 which is incompatible.
fastai 2.7.11 requires torch<1.14,>=1.7, but you have torch 2.0.0 which is incompatible.
Is this related to the release of PyTorch 2.0? If so, how can I fix this? I suppose I need to use torch 1.13.1 but I don’t know how.
Theo
(Theo)
March 16, 2023, 7:48pm
2
Apparently, torchdata and torchvision is working with torch 2.0.0, however, it is not the case for torchtext (or at least I couldn’t figure it out).
If anyone else comes across the same issue, I suggest using torch 1.13.1 if you plan on working with torchtext.
Modifying my code like this solved the problem:
!pip install torch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 torchtext==0.14.1 fastai==2.7.11
!pip install tokenizers
!pip install torchdata==0.5.1
You can check which package is compatible with which torch version from github.
I hope the next version change goes smoother.
1 Like
Are you still running into the issue?
I just tried to install torchtext
in the latest version and I get 0.15.1
as seen here:
>>> import torch
>>> import torchtext
>>> torch.__version__
'2.0.0+cu118'
>>> torchtext.__version__
'0.15.1+cpu'
Theo
(Theo)
March 16, 2023, 10:54pm
4
In the github page, the new 0.15.1
version was not listed, so that’s why I assumed torchtext
wasn’t available to be used with torch 2.0.0
.
I tried installing torchtext 0.15.1
and it works with torch 2.0.0
as you stated.
However, I believe my issue is with torchdata
. When I do !pip install torchdata
it installs the latest 0.6.0
version which has problems with the Multi30k dataset, which may be related to DataLoader.
I’m using the torchdata 0.5.1
now and it seems to be working.
Since torchdata 0.5.1
doesn’t work with torch 2.0.0
, I assume new version of PyTorch is not reliable for people who wants to use torchdata
. But for now, my issue is fixed, I hope they fix the problem with DataLoader in the new version of torchdata
.
Did you already create a GitHub issue for this problem so that the code owners could track and fix it?
Theo
(Theo)
March 16, 2023, 11:45pm
6
I created an issue just now
opened 11:44PM - 16 Mar 23 UTC
### 🐛 Describe the bug
When trying to create an iterator from Multi30k dataset … with the following code:
```
!pip install torch==2.0.0 torchtext==0.15.1
!pip install torchdata==0.6.0
import torch
import torchtext
from torchtext.datasets import Multi30k
SRC_LANGUAGE = 'de'
TGT_LANGUAGE = 'en'
train_iter = Multi30k(split='train', language_pair=(SRC_LANGUAGE, TGT_LANGUAGE))
```
I get the following error:
```
ModuleNotFoundError: Package 'portalocker' is required to be installed to use this datapipe. Please use 'pip install 'portalocker>=2.0.0'' or 'conda install -c conda-forge 'portalocker>=2/0.0' to install the package
```
I then installed the said package like this:
```!pip install 'portalocker>=2.0.0'```
Which solved the issue. And when I tried to iterate over train_iter:
```
for i in train_iter:
print("hello world")
```
I got this error:
```
NameError: name 'portalocker' is not defined
This exception is thrown by __iter__ of _MemoryCellIterDataPipe(remember_elements=1000, source_datapipe=_ChildDataPipe)
```
None of this happens with version 0.5.1 of torchdata.
### Versions
Collecting environment information...
PyTorch version: 2.0.0+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A
OS: Ubuntu 20.04.5 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: 10.0.0-4ubuntu1
CMake version: version 3.22.6
Libc version: glibc-2.31
Python version: 3.9.16 (main, Dec 7 2022, 01:11:51) [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.10.147+-x86_64-with-glibc2.31
Is CUDA available: False
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.7.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 48 bits physical, 48 bits virtual
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 2
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 23
Model: 49
Model name: AMD EPYC 7B12
Stepping: 0
CPU MHz: 2250.000
BogoMIPS: 4500.00
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32 KiB
L1i cache: 32 KiB
L2 cache: 512 KiB
L3 cache: 16 MiB
NUMA node0 CPU(s): 0,1
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Vulnerable
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
Vulnerability Spectre v2: Vulnerable, IBPB: disabled, STIBP: disabled, PBRSB-eIBRS: Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr arat npt nrip_save umip rdpid
Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] torch==2.0.0
[pip3] torchaudio==0.13.1+cu116
[pip3] torchdata==0.6.0
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.15.1
[pip3] torchvision==0.14.1+cu116
[pip3] triton==2.0.0
[conda] Could not collect
1 Like