I’m relative new to PyTorch.
I want to create a Python wheel for a PyTorch extension (in this case GitHub - qwopqwop200/GPTQ-for-LLaMa: 4 bits quantization of LLaMA using GPTQ). But I want to do it using a docker image and with a particular release of PyTorch.
In this particular case I want to compile this extension to be linked with the same libs used in PyTorch 1.13.1 + ROCm 5.2 + Python 3.9 release (Linux). This was installed using pip install mechanism.
I need to use 1.13.1 because 2.x doesn’t work on my Radeon RX5500XT board (memory faults for 2.0, 2.1 and a nightly snapshot I tried).
The above mentioned project contains a C++ code plus hipified kernels. I was able to compile the code using an official AMD docker image containing ROCm 5.2.3 + PyTorch 1.12.1 + Python 3.7. But I need to use the binary in a docker image that has PyTorch 1.13.1 + ROCm 5.2 + Python 3.9 installed. So this wheel is useless.
I want to avoid installing 2 versions of ROCm in the same docker image, so I want to recycle the ROCm libs included with PyTorch 1.3.1. I can use more than one ROCm to create the wheel in a GitHub workflow and then get the wheel as an artifact to install it in the final docker image, but I don’t want to create a 30 GB image with 2 ROCm versions, 2 PyTorch versions, 3 Python versions (like the official AMD images are doing).
I know the PyTorch project uploads docker images with the tools needed to create the PyTorch wheels. I think they are in dockerhub under pytorch/manylinux-builder.
But I’m confused about:
- Which one to download. I see a lot of rocm5.2* tags. What’s the difference? Can I use any of them? Which one was used to generate the PyTorch 1.13.1 release?
- Once I select a docker image: Should I install the PyTorch 1.13.1 + ROCm 5.2 wheels there using pip install?
- How do I install Python 3.9? I think GitHub - pytorch/builder: Continuous builder and binary build scripts for pytorch is using conda to generate the different wheels, is this correct? Where is an example?
Sorry for the long question