[SOLVED] PyTorch no longer supports this GPU because it is too old

Recently, I update the pytorch version to ‘0.3.1’.
I have received the following warning message while running code:
“PyTorch no longer supports this GPU because it is too old.”

What does this mean? The code can not be accelerated using the old GPU.
From now on, all the codes are running only on CPU?

Which GPU do you use?
You could compile from source to get GPU support. The binaries are built for cuda compute capability >= 5 as far as I know.
You can find the instructions here. Let me know if you encounter any problems.

That means that unless you compile pytorch from source all your models will run on the CPU.

See here for instructions on how to compile from source https://github.com/pytorch/pytorch/#from-source

I develop mini model on my pc with Quadro K620 before using some powerful GPU.
I will check the instructions.
Thanks a lot.

I see.
I will look into the instructions.
Thanks a lot.

In my case, even after compiling from the source I still get this error:

Found GPU0 GeForce GT 650M which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.

Any ideas?

@apako Did you managed to get it running? I own a similar card, GeForce GTX 650, also tried to compile it myself (inside a Docker image with nvidia-docker, based on nvidia/cuda:9.0-cudnn7-devel)

Yes, it runs well despite the warning message.

During the build form sources I still get the following error:

...
nvcc fatal   : Unsupported gpu architecture 'compute_30'
...

I thought that during build from sources PyTorch will detect GPU architecture and it builds for it package …

How to solve this issue ?

You can specify the compute capability via TORCH_CUDA_ARCH_LIST=3.0 and try to rebuild it. However, based on the error message, I guess you might be using CUDA>=11.0, which dropped the compute capability 3.0, so you might need to use an older CUDA toolkit.

1 Like

Oh, thanks, I will try :wink:
Yes, I am using CUDA>=11.0 … maybe this is the case …

Yeah, I downgrade the CUDA version to 10.2 and building starts but failed with strange error:

FAILED: confu-deps/NNPACK/src/x86_64-fma/2d-fourier-8x8.py.o 
cd /home/{User}/Projects/AI/pytorch/build/confu-deps/NNPACK && PYTHONPATH=/home/{User}/Projects/AI/pytorch/third_party/python-six:/home/{User}/Projects/AI/pytorch/third_party/python-peachpy /usr/local/bin/python3 -m peachpy.x86_64 -mabi=sysv -g4 -mimage-format=elf -I/home/{User}/Projects/AI/pytorch/third_party/NNPACK/src -I/home/{User}/Projects/AI/pytorch/third_party/NNPACK/src/x86_64-fma -I/home/{User}/Projects/AI/pytorch/third_party/FP16/include -o /home/{User}/Projects/AI/pytorch/build/confu-deps/NNPACK/src/x86_64-fma/2d-fourier-8x8.py.o /home/{User}/Projects/AI/pytorch/third_party/NNPACK/src/x86_64-fma/2d-fourier-8x8.py
/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/writer.py:114: ResourceWarning: unclosed file <_io.FileIO name='/home/{User}/Projects/AI/pytorch/build/confu-deps/NNPACK/src/x86_64-fma/2d-fourier-8x8.py.o' mode='wb' closefd=True>
  self.output_file = None
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Traceback (most recent call last):
  File "/opt/python-3.8.2/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/python-3.8.2/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 280, in <module>
    main()
  File "/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 267, in main
    execute_script(writers, options.input[0])
  File "/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 198, in execute_script
    execute_script(writers, source_filename)
  File "/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 198, in execute_script
    execute_script(writers, source_filename)
  File "/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 202, in execute_script
    exec(code, globals())
  File "/home/{User}/Projects/AI/pytorch/third_party/NNPACK/src/x86_64-fma/2d-fourier-8x8.py", line 5, in <module>
    import fft.complex_soa
  File "/home/{User}/Projects/AI/pytorch/third_party/NNPACK/src/x86_64-fma/fft/complex_soa.py", line 7, in <module>
    from common import cos_npi_over_8, sin_npi_over_8, cos_npi_over_4, sin_npi_over_4
ImportError: cannot import name 'cos_npi_over_8' from 'common' (/home/{User}/.local/lib/python3.8/site-packages/common/__init__.py)
[25/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vlrelu/gen/vlrelu-scalar-x4.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vunary.h:11,
                 from ../third_party/XNNPACK/src/f32-vlrelu/gen/vlrelu-scalar-x4.c:13:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[26/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vrnd/gen/vrndd-scalar-libm-x1.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vunary.h:11,
                 from ../third_party/XNNPACK/src/f32-vrnd/gen/vrndd-scalar-libm-x1.c:15:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[27/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-velu/gen/velu-scalar-rr2-p6-x6.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vunary.h:11,
                 from ../third_party/XNNPACK/src/f32-velu/gen/velu-scalar-rr2-p6-x6.c:14:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[28/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vmulcaddc/gen/c2-minmax-scalar-2x.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vmulcaddc.h:11,
                 from ../third_party/XNNPACK/src/f32-vmulcaddc/gen/c2-minmax-scalar-2x.c:13:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[29/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vrnd/gen/vrndd-scalar-libm-x2.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vunary.h:11,
                 from ../third_party/XNNPACK/src/f32-vrnd/gen/vrndd-scalar-libm-x2.c:15:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[30/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vmulcaddc/gen/c4-minmax-scalar-2x.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vmulcaddc.h:11,
                 from ../third_party/XNNPACK/src/f32-vmulcaddc/gen/c4-minmax-scalar-2x.c:13:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[31/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vrnd/gen/vrndd-scalar-libm-x4.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vunary.h:11,
                 from ../third_party/XNNPACK/src/f32-vrnd/gen/vrndd-scalar-libm-x4.c:15:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "setup.py", line 773, in <module>
    build_deps()
  File "setup.py", line 315, in build_deps
    build_caffe2(version=version,
  File "/home/{User}/Projects/AI/pytorch/tools/build_pytorch_libs.py", line 58, in build_caffe2
    cmake.build(my_env)
  File "/home/{User}/Projects/AI/pytorch/tools/setup_helpers/cmake.py", line 346, in build
    self.run(build_args, my_env)
  File "/home/{User}/Projects/AI/pytorch/tools/setup_helpers/cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "/opt/python-3.8.2/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '8']' returned non-zero exit status 1.

Error is:

ImportError: cannot import name 'cos_npi_over_8' from 'common'

But how it’s possible, all submodules updated properly …

This error seems to be raised by NNPACK and while I’m not deeply familiar with it, I would recommend to uninstall all previously installed PyTorch versions, clear the build via python setup.py clean, checkout all sumbodules, and try to rebuild it.

1 Like

Another strange issue:

-- Using third party subdirectory Eigen.
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.8.5", minimum required is "3.0") 
-- Found PythonLibs: /usr/lib/libpython3.8.so.1.0 (Required is at least version "3.0") 
CMake Error at cmake/Dependencies.cmake:959 (if):
  if given arguments:

    "VERSION_LESS" "3"

  Unknown arguments specified
Call Stack (most recent call first):
  CMakeLists.txt:549 (include)


-- Configuring incomplete, errors occurred!

I just hard-coded version of the Python that was detected wrong to 3.8

And then build starts but faced with another issue:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/redra/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 280, in <module>
    main()
  File "/home/redra/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 267, in main
[ 57%] Built target XNNPACK
    execute_script(writers, options.input[0])
  File "/home/redra/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 198, in execute_script
    execute_script(writers, source_filename)
  File "/home/redra/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 198, in execute_script
    execute_script(writers, source_filename)
  File "/home/redra/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 202, in execute_script
    exec(code, globals())
  File "/home/redra/Projects/AI/pytorch/third_party/NNPACK/src/x86_64-fma/2d-fourier-8x8.py", line 5, in <module>
    import fft.complex_soa
  File "/home/redra/Projects/AI/pytorch/third_party/NNPACK/src/x86_64-fma/fft/complex_soa.py", line 7, in <module>
    from common import cos_npi_over_8, sin_npi_over_8, cos_npi_over_4, sin_npi_over_4
ImportError: cannot import name 'cos_npi_over_8' from 'common' (/home/redra/.local/lib/python3.8/site-packages/common/__init__.py)
[ 57%] Building C object sleef/src/libm/CMakeFiles/sleefavx512fnofma.dir/sleefsimddp.c.o
[ 57%] Building CXX object caffe2/proto/CMakeFiles/Caffe2_PROTO.dir/caffe2.pb.cc.o
make[2]: *** [confu-deps/NNPACK/CMakeFiles/nnpack.dir/build.make:81: confu-deps/NNPACK/src/x86_64-fma/2d-fourier-8x8.py.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:3282: confu-deps/NNPACK/CMakeFiles/nnpack.dir/all] Error 2

Seems like this is related to sudo make install -- ImportError: cannot import name cos_npi_over_8 · Issue #2296 · facebookarchive/caffe2 · GitHub

Even build from docker failed:

#23 325.0 [  8%] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/shuffle/jit_uni_shuffle.cpp.o
#23 337.2 [  8%] Built target dnnl_cpu_x64
#23 337.2 make: *** [all] Error 2
#23 337.2 Makefile:140: recipe for target 'all' failed
#23 337.2 Building wheel torch-1.8.0a0
#23 337.2 -- Building version 1.8.0a0
#23 337.2 cmake -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/opt/pytorch/torch -DCMAKE_PREFIX_PATH=/opt/conda/bin/../ -DNUMPY_INCLUDE_DIR=/opt/conda/lib/python3.8/site-packages/numpy/core/include -DPYTHON_EXECUTABLE=/opt/conda/bin/python -DPYTHON_INCLUDE_DIR=/opt/conda/include/python3.8 -DPYTHON_LIBRARY=/opt/conda/lib/libpython3.8.so.1.0 -DTORCH_BUILD_VERSION=1.8.0a0 -DUSE_NUMPY=True /opt/pytorch
#23 337.2 cmake --build . --target install --config Release -- -j 8
#23 337.2 Traceback (most recent call last):
#23 337.2   File "setup.py", line 773, in <module>
#23 337.2     build_deps()
#23 337.2   File "setup.py", line 315, in build_deps
#23 337.2     build_caffe2(version=version,
#23 337.2   File "/opt/pytorch/tools/build_pytorch_libs.py", line 58, in build_caffe2
#23 337.2     cmake.build(my_env)
#23 337.2   File "/opt/pytorch/tools/setup_helpers/cmake.py", line 345, in build
#23 337.2     self.run(build_args, my_env)
#23 337.2   File "/opt/pytorch/tools/setup_helpers/cmake.py", line 140, in run
#23 337.2     check_call(command, cwd=self.build_dir, env=env)
#23 337.2   File "/opt/conda/lib/python3.8/subprocess.py", line 364, in check_call
#23 337.2     raise CalledProcessError(retcode, cmd)
#23 337.2 subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '8']' returned non-zero exit status 2.
------
error committing ki40ko690hn6nxbbd83w6smqr: invalid mutable ref 0xc001c50fa0: invalid: executor failed running [/bin/sh -c TORCH_CUDA_ARCH_LIST="3.0 3.5 5.2 6.0 6.1 7.0+PTX 8.0" TORCH_NVCC_FLAGS="-Xfatbin -compress-all"     CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"     python setup.py install]: exit code: 1
make: *** [docker.Makefile:47: devel-image] Error 1

I would recommend to create a new conda or pip environment with the wanted Python version and try to reinstall it in this clean environment.

1 Like

I also tried to build on WIndows:

  Stringify NVFUSER runtime source file
  Building Custom Rule C:/pytorch/caffe2/CMakeLists.txt
  Stringify NVFUSER runtime source file
  Building Custom Rule C:/pytorch/caffe2/CMakeLists.txt
  Building NVCC (Device) object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/THC/Release/torch_cuda_generated_THCReduce
  ApplyUtils.cu.obj
  nvcc fatal   : Unsupported gpu architecture 'compute_30'
  CMake Error at torch_cuda_generated_THCReduceApplyUtils.cu.obj.Release.cmake:221 (message):
    Error generating
    C:/pytorch/build/caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/THC/Release/torch_cuda_generated_THCReduceApplyUtils.
  cu.obj

compute_30, but I have set TORCH_CUDA_ARCH_LIST=3.0 … seems like another variable should be set for caffe2

This error is raised, since the CUDA compiler nvcc doesn’t recognize the GPU architecture. As explained before, CUDA>=11.0 dropped the compute capability 3.0, so you would have to use an older CUDA version.

I’ve found the root cause for this issue: