Hello all,
I have a small question regarding installing Torch on Linux, in particular on Ubuntu/Ubuntu-based distributions. It’s mostly a Python/pip management question, still strongly related to Torch, thus I think it’s a good point to ask here (and I hope I selected an appropriate category).
On Windows, usually my globally installed Python packages are all organized by pip3 install (maybe extended by a few preinstalled ones). However on Linux, some of them are installed by the package manager, either explicitly by the user or implicitly as a dependency of installed programs. Therefore, I always try to install the required Python packages using the package manager (i.e. apt on Ubuntu), and of course avoid global pip installations. This works fine with almost all packages (numpy, pandas, etc.). Now, Torch is a special case here. First, it is not available by a apt PPA (i.e. installation requires pip) and it has a quite a lot of dependencies that could be provided partially by the system (e.g. numpy), while others could still be required but are not installed so far.
Clearly, the go-to solution on Linux is using virtual envorionments and install Torch inside one, which works fine. However using multiple virtual environments or using multiple users on the same machine, this approach installs Torch multiple times (at least to the best of my knowledge). Thus, I am looking for a solution to install Torch globally using pip, but to use the provided System packages (e.g. numpy) as much as possible to avoid redundancies and incompatabilities. To avoid inferences with system packages, a valid installation location is usr/local/lib/pythonX.XX (at least on Ubuntu and derived distros), i.e. I use:
sudo pip3 install --target '/usr/local/lib/python3.12/dist-packages' <list of packages to install>
Installing Torch this way works, but there are some issues: It installs numpy-2.X, whereas Ubuntu’s python3-numpy Debian package is still numpy-1.X. To the best of my knowledge, there are also Torch builds available that work with numpy-1.X.
Therefore, I would like to use some pip3 install command with some option try to use the existing debian packages and only install dependencies that are not resolved by those to avoid conflicts like parallel installing numpy-1 (by apt to /usr/lib/python3) and numpy-2 (by pip to /usr/local/lib/python3.12). In case of incompatabile selections (e.g. preinstalled numpy is 1.X whereas some package requires 2.X and is not compatible with 1.X), either install the dependency or exit with an error (I’d prefer the error). This would be a great way of maintining system-wide Python packages (in particular Torch) parallel to the system-wide ones installed by the system’s package manager.
Is this possible somehow?
Appreciating any comments and thanks in advance!