ERROR LNK2019 unresolved external symbol ""float __cdecl pow(float,int)" in function ""void __cdecl test_atomic_mul<float>(void)"

Oh ok, sorry I am fairly new to all of this.

I used the x64 Native Tools Command Prompt for VS 2019, but I did run it without DISTUTILS_USE_SDK=1.
Nonetheless, the command python setup.py build --cmake has apparently finished succesfully, and the error only occurs during python setup.py install (based on the CMakeError.log, I think it is the same error as before).
I did not yet use the logging technique you mentioned above, but I just copied and pasted the console output. I am sharing this along with the CMakeLogs in the hope, that these are of any use in this specific case. If not, please let me know, then I will rebuild with a ā€œfull logā€ as you described it above.

Thanks for suggesting Ninja, it appears to work slightly better than the VS Generator, and I will stick to that from now.
Note that I made a backup of the whole pytorch-1.7.0 folder after python setup.py build --cmake, so if the issue is only related to the actual installation I could probably just use that again instead of doing the whole build again.

@peterjc123 are the provided logs of any help?
If not, should I use > build.log 2>&1 when building with Ninja or the VC Generator?

Many thanks in advance!

Please remove all the files in C:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build and try again.

As I have tried several different approaches in the meantime, I am a little confused about which directory exactly you refer to, as well as what you mean by trying again.
I assume, you are referring to the one generated by building with Ninja, and mean, I should run python setup.py build --cmake again?

Note that I am currently building with VS Generator again in order to obtain a ā€œfull logā€. Trying the same with Ninja earlier caused some issues again, and I did not want to tackle even more problems, when they are avoidable :slight_smile: .

I should mention, that I made a backup of a freshly synced & reset pytorch-1.7.0 directory where I included the 4 manual fixes mentioned above, and I am just copying this backup to gooog/Downloads when retrying to build. I think this should be fine, but I wanted to mention it just in case the issue could be related to that.

Just for clarification: Is there a difference between starting the whole process again (copying said backup and running the installation routine) and deleting the pytorch-1.7.0/build directory? E. g. does python setup.py build --cmake change anything else in the pytorch-1.7.0 directory than generating the /build directory?

The VS Generator build finished now, once again with similar errors, but this time I saved the output to a log file as you explained:
Full build.log

As I said in my last post, I am right now a little bit confused as of what the next steps are, because I feel like we are treating several different problems / methods at the same time. I would be very thankful, if you could elaborate a little bit on what I should do next.
Anyways thank you very much for your support, I appreciate it a lot, and I hope that this log can help you helping me :slight_smile: .

Accroding to your log, the error is caused by the fact that NVCC cannot find MSVC.

  Building NVCC (Device) object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/THC/Release/torch_cuda_generated_THCReduceApplyUtils.cu.obj
  -- Removing C:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/THC/Release/torch_cuda_generated_THCReduceApplyUtils.cu.obj
  C:/Users/gooog/anaconda3/envs/torchbase/Library/bin/cmake.exe -E remove C:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/THC/Release/torch_cuda_generated_THCReduceApplyUtils.cu.obj
  -- Generating dependency file: C:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/THC/torch_cuda_generated_THCReduceApplyUtils.cu.obj.NVCC-depend
  "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/bin/nvcc.exe" -M -D__CUDACC__ C:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/aten/src/THC/THCReduceApplyUtils.cu -o C:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/THC/torch_cuda_generated_THCReduceApplyUtils.cu.obj.NVCC-depend -ccbin "C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30037/bin/HostX64/x64/cl.exe" -m64 -Dtorch_cuda_EXPORTS -DTORCH_CUDA_BUILD_MAIN_LIB -DTH_BLAS_MKL -D_OPENMP_NOFORCE_MANIFEST -DONNX_ML=1 -DONNXIFI_ENABLE_EXT=1 -DONNX_NAMESPACE=onnx_torch -D_CRT_SECURE_NO_DEPRECATE=1 -DWIN32_LEAN_AND_MEAN -DIDEEP_USE_MKL -DUSE_EXTERNAL_MZCRC -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -Xcompiler ,\"/DWIN32\",\"/D_WINDOWS\",\"/GR\",\"/EHsc\",\"/w\",\"/MP\",\"/bigobj\",\"-openmp:experimental\",\"-DNDEBUG\",\"-DUSE_FBGEMM\",\"-DUSE_VULKAN_WRAPPER\",\"-DHAVE_AVX_CPU_DEFINITION\",\"-DHAVE_AVX2_CPU_DEFINITION\",\"/MD\",\"/O2\",\"/Ob2\",\"/DNDEBUG\",\"/w\",\"/MP\",\"/bigobj\",\"-DNDEBUG\" -Xcompiler /w -w -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch -gencode arch=compute_35,code=sm_35 -Xcudafe --diag_suppress=cc_clobber_ignored -Xcudafe --diag_suppress=integer_sign_change -Xcudafe --diag_suppress=useless_using_declaration -Xcudafe --diag_suppress=set_but_not_used -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=implicit_return_from_non_void_function -Xcudafe --diag_suppress=unsigned_compare_with_zero -Xcudafe --diag_suppress=declared_but_not_referenced -Xcudafe --diag_suppress=bad_friend_decl --Werror cross-execution-space-call --no-host-device-move-forward -Xcompiler -MD --expt-relaxed-constexpr --expt-extended-lambda -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/win_tmp/tmp_includes -Xcompiler /wd4819 -Xcompiler /wd4503 -Xcompiler /wd4190 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4275 -Xcompiler /wd4522 -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DNVCC "-IC:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/include" -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/aten/src -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/aten/src -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0 -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/cmake/../third_party/googletest/googlemock/include -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/cmake/../third_party/googletest/googletest/include -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/third_party/protobuf/src -IC:/Users/gooog/Downloads/mkl/include -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/cmake/../third_party/benchmark/include -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/third_party -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/cmake/../third_party/eigen -IC:/Users/gooog/anaconda3/envs/torchbase/include -IC:/Users/gooog/anaconda3/envs/torchbase/lib/site-packages/numpy/core/include "-IC:/Program Files (x86)/Python38-32/include" -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/cmake/../third_party/cub -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/caffe2/contrib/aten -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/third_party/onnx -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/third_party/onnx -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/third_party/foxi -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/third_party/foxi -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/third_party/ideep/mkl-dnn/include -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/third_party/ideep/include -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/caffe2/aten/src/TH -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/aten/src/TH -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/caffe2/aten/src/THC -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/aten/src/THC -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/aten/src/THCUNN -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/aten/src/ATen/cuda -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/caffe2/aten/src -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/aten/../third_party/catch/single_include -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/aten/src/ATen/.. -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/caffe2/aten/src/ATen -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/c10/cuda/../.. -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/c10/../ "-IC:/Program Files/NVIDIA Corporation/NvToolsExt/include" -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/torch/csrc/api -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/torch/csrc/api/include -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/third_party/ideep/mkl-dnn/include -IC:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/third_party/ideep/mkl-dnn/src/../include
  THCReduceApplyUtils.cu
  Der Befehl "cl.exe" ist entweder falsch geschrieben oder
  konnte nicht gefunden werden.
  CMake Error at torch_cuda_generated_THCReduceApplyUtils.cu.obj.Release.cmake:221 (message):
    Error generating
    C:/Users/gooog/Downloads/Pytorch/pytorch-1.7.0/build/caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/THC/Release/torch_cuda_generated_THCReduceApplyUtils.cu.obj

Are you trying to trigger a build in a MSVC dev shell? For the VS generator, please do that in a normal shell. If CUDA still fails to find CL, then please set CUDAHOSTCXX.

python setup.py build --cmake
python setup.py install

could be combined into one single command:

python setup.py install

If you donā€™t do that, the Ninja generator may not accept that because it tries to trigger a rebuild, which is broken.

Well, itā€™s better to do a clean build. Deleting build sometimes may not be enough.

I was using the VS shell you recommended here:

But I think there was some error when I fixed the program to my task bar, where it did not start properly. The first output used to be

Der Befehl "C:\Program" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.

C:\WINDOWS\system32>

And I could confirm with where cl.exe that the shell can indeed not find the program.

When opening the shell correctly, the first lines look a lot better, and it can find cl.exe:

**********************************************************************
** Visual Studio 2019 Developer Command Prompt v16.10.0
** Copyright (c) 2021 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

C:\Program Files (x86)\Microsoft Visual Studio\2019\Community>where cl.exe
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\Hostx64\x64\cl.exe

C:\Program Files (x86)\Microsoft Visual Studio\2019\Community>

On the other hand, the latter message also ocurred a bit later in the ā€œbrokenā€ prompt, so I will use the standart Command Prompt from now on.

I switched to the separated version, because I thought it would help determining, wether the problem is related to building or installation. But I will use python setup.py install then. Is --cmake redundant?

Ok, that makes sense. Thank you.

I will now try to build again, using the same routine as I did when generating the full log, but using the Standart Command Prompt and only python setup.py install. The full routine is thus:

cd C:\Users\gooog\Downloads
cd Pytorch
cd pytorch-1.7.0
conda create --name gputorch
y
conda activate gputorch
conda install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests dataclasses
y
conda install -c conda-forge libuv=1.39
y

set TMP_DIR_WIN=C:\Users\gooog\Downloads
C:/Users/gooog/Downloads/Use_mkl.bat > build.log 2>&1
C:/Users/gooog/Downloads/Use_sccache.bat > build.log 2>&1
set USE_NINJA=OFF
set CMAKE_VERBOSE_MAKEFILE=1
set CMAKE_GENERATOR=Visual Studio 16 2019
set CMAKE_GENERATOR_TOOLSET_VERSION=14.29
set DISTUTILS_USE_SDK=1
set PATH=%TMP_DIR_WIN%\bin;%PATH%
sccache --stop-server > build.log 2>&1
sccache --start-server > build.log 2>&1
sccache --zero-stats > build.log 2>&1
set CC=sccache-cl
set CXX=sccache-cl
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%
set CUDAHOSTCXX=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\HostX64\x64\cl.exe
set MAX_JOBS=1
set USE_GLOG=0
set USE_GLOO=0
python setup.py install > build.log 2>&1

Hopfully, it will work this time.

During build, I encountered yet another error:

New build.log

I build using exactly the settings above and the standart Command Prompt. Any idea on how to fix this?

After doing some research, I think this is another case of this intermittent error, with which you should already be familiar :slight_smile: .
I therefore created a C:/Users/gooog/Downloads/Downloads2 directory, and I am currently just trying to build again there. I did this because I wanted to preserve the older one in case there is still some valuable information to obtain from it. I also redownloaded the pytorch-1.7.0 repo. My full installation routine is:


cd C:\Users\gooog\Downloads\Downloads2
mkdir Pytorch
cd Pytorch
conda create --name pytorch_gpu_env
y
conda activate pytorch_gpu_env
conda install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests dataclasses
y
conda install -c conda-forge libuv=1.39
y
git clone --branch v1.7.0 https://github.com/pytorch/pytorch.git pytorch-1.7.0
cd pytorch-1.7.0
git clean -xdfcond
python setup.py clean
git submodule deinit -f --all
git submodule sync --recursive
git submodule update --init --recursive

##################################
#   FIX const ERROR
#   FIX floor ERROR
#   FIX cuda_atomic_ops_test ERROR
#   FIX lnk ERROR
##################################

set TMP_DIR_WIN=C:\Users\gooog\Downloads\Downloads2
C:/Users/gooog/Downloads/Downloads2/Use_mkl.bat > build.log 2>&1
C:/Users/gooog/Downloads/Downloads2/Use_sccache.bat > build.log 2>&1
set USE_NINJA=OFF
set CMAKE_VERBOSE_MAKEFILE=1
set CMAKE_GENERATOR=Visual Studio 16 2019
set CMAKE_GENERATOR_TOOLSET_VERSION=14.29
set DISTUTILS_USE_SDK=1
set PATH=%TMP_DIR_WIN%\bin;%PATH%
sccache --stop-server > build.log 2>&1
sccache --start-server > build.log 2>&1
sccache --zero-stats > build.log 2>&1
set CC=sccache-cl
set CXX=sccache-cl
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%
set CUDAHOSTCXX=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\HostX64\x64\cl.exe
set MAX_JOBS=1
set USE_GLOG=0
set USE_GLOO=0
python setup.py install > build.log 2>&1

@peterjc123 The second build resulted in exactly the same error. I have already encountered this problem earlier, and am now of the opinion, that my setup is somehow able to reproduce the ā€œintermittent errorā€ mentioned above consistently. Any idea on how to fix this?
This is the newest build2.log.

Sorry for the late reply. Itā€™s an intermittent one. Just retry the build a few times.

1 Like

I finally got it to work!
Thank you very much for your help, I appreciate it a lot!