Force installing torchvision (on IBM ppc64le)

1

I am in a weird scenario were I am forced to use torch 1.3.1 (due to hardware see: https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/). I read from the pytorch docs that it’s corresponding version of torchvision is 0.4.1 (torchvision · PyPI):

Installation

We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.

Installation
We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.

torch	torchvision	python
master / nightly	master / nightly	>=3.6
1.5.0	0.6.0	>=3.5
1.4.0	0.5.0	==2.7, >=3.5, <=3.8
1.3.1	0.4.2	==2.7, >=3.5, <=3.7
1.3.0	0.4.1	==2.7, >=3.5, <=3.7
1.2.0	0.4.0	==2.7, >=3.5, <=3.7
1.1.0	0.3.0	==2.7, >=3.5, <=3.7
<=1.0.1	0.2.2	==2.7, >=3.5, <=3.7

but for some reason I have the wrong version of it:

torchvision 0.2.2 pypi_0 pypi

is there a way to install the right version of torchvision?

What I’ve tried:

First I tried force installing the right version with conda. Conda couldn’t find the version of torchvision that I need:

$ conda install torchvision==0.4.2
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - torchvision==0.4.2

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Then I proceeded to try to install it regardless with pip

$ pip install torchvision==0.4.2
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement torchvision==0.4.2 (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3)
ERROR: No matching distribution found for torchvision==0.4.2

got an error too.

Is there anything else to try?

related:

2

Wouldn’t it be more natural to use the torchvision 0.4.2 from the same page?

3

I tried pip install torchvision==0.4.2 and conda install torchvision==0.4.2 but it didn’t install them.

hmmm…can you clarify how to actually do “getting them from that same page”? Thats what I thought too but not sure how. @tom

Btw, thanks for the help.

4

about to try

conda install torchvision==0.4.2 -c pytorch
5

The IBM page you linked lists TorchVision alongside PyTorch, so I would expect you to be able to install PyTorch and TorchVision by following their instructions (using IBM’s channel or something).

1 Like
6

Oh great point will check that link again and see what happens.

7

what instructions are you referring to?

8

Under the documentation tab on that web page
https://www.ibm.com/support/knowledgecenter/SS5SF7_1.7.0/navigation/wmlce_install.htm

9

Oh yea I read that. I’m running out of ideas what I need to run from that page for things to work.

10

hmmm… I did read it. What doesn’t make sense to me is that they suggest to force the release so I did:

$ conda install torchvision powerai-release=1.7.0
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - powerai-release=1.7.0
  - torchvision

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

but then when I check their website for that powerai-release it does have the 1.7.0 and the 0.4.2 torchvision version I want but it can’t find it.

Indeed it is not in the list:

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

so what I need is to be able to point it to this channel conda
https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/

…

alternatively that website allows me to download the tar but I dont know how to get conda to install from that. @tom

11

It tells you to use this for a channel:

https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/

by running

conda config --prepend channels \
https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
12

where did it say that? (I’ve been looking for that command)

Will try right now.

Thanks for the help Tom.

13

@tom

I did prepend the conda channel:

$ conda config --prepend channels \
> https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/

but are you sure that is the right URL address? I get the following error:

$ conda install torchvision powerai-release=1.7.0

Collecting package metadata (current_repodata.json): failed

CondaHTTPError: HTTP 500 INTERNAL SERVER ERROR for url <https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/linux-ppc64le/current_repodata.json>
Elapsed: 00:00.706870

A remote server error occurred when trying to retrieve this URL.

A 500-type error (e.g. 500, 501, 502, 503, etc.) indicates the server failed to
fulfill a valid request.  The problem may be spurious, and will resolve itself if you
try your request again.  If the problem persists, consider notifying the maintainer
of the remote server.
14

I don’t know, at some point you should probably check with IBM.
The URL with the 500 does work when I open it locally.

15

thanks tom.

I am working on it with them https://github.com/IBM/powerai/issues/268 and with the sys admin on my cluster. Will paste a solution once I get it. I keep getting issues but I don’t think they have to do with you so I’m unsure if pytorch peeps can help without getting more involved.

I will post my solution here once I get it regardless.

16

For all details check (https://github.com/IBM/powerai/issues/268).

Make sure you have the right conda channel prepended:

conda config --prepend channels https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/

then install the powerai wmlce that you want e.g. 1.7.0 (most recent as of this writing):

conda create -n my_new_env python=3.7 powerai=1.7.0
conda activate my_new_env
17

in my special case my HPC has conda envs I can clone btw.