How does one install PyTorch and related tools from within the setup.py install_requires list?

I was trying to test the pip install -e . on my library ultimate-utils after changing laptops but I keep getting errors of this type:

(uutils_env) brandomiranda~/ultimate-utils ❯ pip install -e .                      
Obtaining file:///Users/brandomiranda/ultimate-utils
  Preparing metadata (setup.py) ... done
ERROR: Could not find a version that satisfies the requirement torchvision==0.10.1 (from ultimate-utils) (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3)
ERROR: No matching distribution found for torchvision==0.10.1

where it seems it stopped working to install anything related to pytorch for some reason.

I tried updating pip and conda but it did not work. I did:

pip install --upgrade pip
conda update conda
conda update conda-build
conda update -n base -c defaults conda
conda update --name base conda
conda update --all
conda install anaconda

but none seem to work.

my setup.py looks as follows:

"""
conda create -n uutils_env python=3.9
conda activate uutils_env
conda remove --all --name uutils_env
rm -rf /Users/brando/anaconda3/envs/uutils_env

pip install -e ~/ultimate-utils/ultimate-utils-proj-src/

pip install ultimate-utils

To test it:
python -c "import uutils; uutils.hello()"
python -c "import uutils; uutils.torch_uu.hello()"

python -c "import uutils; uutils.torch_uu.gpu_test_torch_any_device()"
python -c "import uutils; uutils.torch_uu.gpu_test()"

PyTorch:
    basing the torch install from the pytorch website as of this writing: https://pytorch.org/get-started/locally/
    pip3 install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html

refs:
    - setup tools: https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#using-find-or-find-packages
"""
from setuptools import setup
from setuptools import find_packages
import os

# import pathlib

here = os.path.abspath(os.path.dirname(__file__))
with open(os.path.join(here, 'README.md'), encoding='utf-8') as f:
    long_description = f.read()

setup(
    name='ultimate-utils',  # project name
    version='0.5.3',
    description="Brando's ultimate utils for science, machine learning and AI",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url='https://github.com/brando90/ultimate-utils',
    author='Brando Miranda',
    author_email='brandojazz@gmail.com',
    python_requires='>=3.9.0',
    license='MIT',
    package_dir={'': 'ultimate-utils-proj-src'},
    packages=find_packages('ultimate-utils-proj-src'),  # imports all modules/folders with  __init__.py & python files

    # for pytorch see doc string at the top of file
    install_requires=[
        # 'torch==1.9.1',
        'torchvision==0.10.1',
        'torchaudio==0.9.1',
        'dill',
        'networkx>=2.5',
        'scipy',
        'scikit-learn',
        'lark-parser',
        'torchtext==0.10.1',
        'tensorboard',
        'pandas',
        'progressbar2',
        'transformers',
        'requests',
        'aiohttp',
        'numpy',
        'plotly',
        'wandb',
        'matplotlib',
        # 'seaborn'

        # 'pygraphviz'  # removing because it requires user to install graphviz and gives other issues
    ]
)

the dir structure is simple:

enter image description here

How do I fix this and what is not working?

I am trying to avoid having to run pip commands outside like:

conda install pytorch torchvision torchaudio -c pytorch

at least for cpu in my local laptop…if I can install gpu ones automatically with a flag it could be nice but for future work…


Related resources I tried:

Hi,

Given the list of available versions that it shows, my first guess is that it is not a supported platform and so the only binaries that are valid are (very) old universal binaries that we use to provide (by mistake).

What is your machine, python version, os, etc?

I was using python 3.9, mac os X, macbook pro, let me think what else and try without those equalities…

I am using torchmeta: pytorch-meta/setup.py at master Β· tristandeleu/pytorch-meta Β· GitHub that has

install_requires=[
        'torch>=1.4.0,<1.10.0',
        'torchvision>=0.5.0,<0.11.0',
        'numpy>=1.14.0',
        'Pillow>=7.0.0',
        'h5py',
        'tqdm>=4.0.0',
        'requests',  # Required by Torchvision
        'ordered-set'
    ],

@albanD was this suppose to fail too?

    install_requires=[
        'torch>=1.4.0,<1.10.0',
        'torchvision>=0.5.0,<0.11.0',
        'torchaudio>=0.9.1',

error:

(uutils_env) brandomiranda~/ultimate-utils ❯ pip install -e .
Obtaining file:///Users/brandomiranda/ultimate-utils
  Preparing metadata (setup.py) ... done
ERROR: Could not find a version that satisfies the requirement torch<1.10.0,>=1.4.0 (from ultimate-utils) (from versions: none)
ERROR: No matching distribution found for torch<1.10.0,>=1.4.0

why is it saying

(from versions: none)

??


I think it’s the python 3.10 version that causes the issues, bizarre, why is that…? Saying that since the current installation seems to be working:

(meta_learning) brandomiranda~ ❯ conda create -n uutils_env python=3.9
conda activate uutils_env
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /Users/brandomiranda/miniconda3/envs/uutils_env

  added / updated specs:
    - python=3.9


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    certifi-2021.10.8          |   py39hecd8cb5_0         151 KB
    pip-21.2.4                 |   py39hecd8cb5_0         1.8 MB
    python-3.9.7               |       h88f2d9e_1         9.8 MB
    setuptools-58.0.4          |   py39hecd8cb5_0         792 KB
    ------------------------------------------------------------
                                           Total:        12.5 MB

The following NEW packages will be INSTALLED:

  ca-certificates    pkgs/main/osx-64::ca-certificates-2021.10.26-hecd8cb5_2
  certifi            pkgs/main/osx-64::certifi-2021.10.8-py39hecd8cb5_0
  libcxx             pkgs/main/osx-64::libcxx-12.0.0-h2f01273_0
  libffi             pkgs/main/osx-64::libffi-3.3-hb1e8313_2
  ncurses            pkgs/main/osx-64::ncurses-6.3-hca72f7f_2
  openssl            pkgs/main/osx-64::openssl-1.1.1l-h9ed2024_0
  pip                pkgs/main/osx-64::pip-21.2.4-py39hecd8cb5_0
  python             pkgs/main/osx-64::python-3.9.7-h88f2d9e_1
  readline           pkgs/main/osx-64::readline-8.1-h9ed2024_0
  setuptools         pkgs/main/osx-64::setuptools-58.0.4-py39hecd8cb5_0
  sqlite             pkgs/main/osx-64::sqlite-3.36.0-hce871da_0
  tk                 pkgs/main/osx-64::tk-8.6.11-h7bc2e8c_0
  tzdata             pkgs/main/noarch::tzdata-2021e-hda174b7_0
  wheel              pkgs/main/noarch::wheel-0.37.0-pyhd3eb1b0_1
  xz                 pkgs/main/osx-64::xz-5.2.5-h1de35cc_0
  zlib               pkgs/main/osx-64::zlib-1.2.11-h1de35cc_3


Proceed ([y]/n)? 


Downloading and Extracting Packages
pip-21.2.4           | 1.8 MB    | ########################################################################################################################################## | 100% 
python-3.9.7         | 9.8 MB    | ########################################################################################################################################## | 100% 
setuptools-58.0.4    | 792 KB    | ########################################################################################################################################## | 100% 
certifi-2021.10.8    | 151 KB    | ########################################################################################################################################## | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate uutils_env
#
# To deactivate an active environment, use
#
#     $ conda deactivate

then the pip install of my lib

(uutils_env) brandomiranda~/ultimate-utils ❯ pip install -e .
Obtaining file:///Users/brandomiranda/ultimate-utils
Collecting dill
  Using cached dill-0.3.4-py2.py3-none-any.whl (86 kB)
Collecting networkx>=2.5
  Using cached networkx-2.6.3-py3-none-any.whl (1.9 MB)
Collecting scipy
  Downloading scipy-1.7.3-cp39-cp39-macosx_10_9_x86_64.whl (33.2 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 33.2 MB 56 kB/s 
Collecting scikit-learn
  Downloading scikit_learn-1.0.1-cp39-cp39-macosx_10_13_x86_64.whl (8.0 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 8.0 MB 51 kB/s 
Collecting lark-parser
  Using cached lark_parser-0.12.0-py2.py3-none-any.whl (103 kB)
Collecting torchtext==0.10.1
  Downloading torchtext-0.10.1-cp39-cp39-macosx_10_9_x86_64.whl (1.5 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1.5 MB 76 kB/s 
Collecting tensorboard
  Downloading tensorboard-2.7.0-py3-none-any.whl (5.8 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 5.8 MB 97 kB/s 
Collecting pandas
  Downloading pandas-1.3.4-cp39-cp39-macosx_10_9_x86_64.whl (11.6 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 11.6 MB 88 kB/s 
Collecting progressbar2
  Downloading progressbar2-3.55.0-py2.py3-none-any.whl (26 kB)
Collecting transformers
  Downloading transformers-4.13.0-py3-none-any.whl (3.3 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3.3 MB 58 kB/s 
Collecting requests
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 62 kB 117 kB/s 
Collecting aiohttp
  Downloading aiohttp-3.8.1-cp39-cp39-macosx_10_9_x86_64.whl (574 kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 574 kB 26 kB/s 
Collecting numpy
  Downloading numpy-1.21.4-cp39-cp39-macosx_10_9_x86_64.whl (17.0 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 17.0 MB 68 kB/s 
Collecting plotly
  Downloading plotly-5.4.0-py2.py3-none-any.whl (25.3 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 25.3 MB 55 kB/s 
Collecting wandb
  Downloading wandb-0.12.7-py2.py3-none-any.whl (1.7 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1.7 MB 64 kB/s 
Collecting matplotlib
  Downloading matplotlib-3.5.0-cp39-cp39-macosx_10_9_x86_64.whl (7.3 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 7.3 MB 49 kB/s 
Collecting torch<1.10.0,>=1.4.0
  Downloading torch-1.9.1-cp39-none-macosx_10_9_x86_64.whl (218.8 MB)
     |β–ˆβ–ˆβ–ˆ                             | 21.0 MB 66 kB/s eta 0:49:20

I think I have those version since those are the ones I assume worked in my cluster since I have this command in my notes:

    pip3 install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html

It seems that once you sort out your PATH (which mine was broken details Why is python using 3.8.1 and 3.9, then fail to install packages (ERROR: Package pkg requires a different Python: 3.8.1 not in '>=3.9.0')? - Stack Overflow) then create a conda env with python 3.9 things seemed to work. Somehow Python 3.10 might be too new and borken.

1 Like