Install from src master bombing on Win10/Anaconda 5.x

I have been trying to install the latest PyTorch from source on Win10, and have yet make it work. At least a couple things are going wrong:

  1. It cannot find MKL or BLAS, which I know is included in Anaconda: I can find the libraries manually. I’m afraid I’m not familiar with CMake, and so I’m not quite sure how to add them. I tried setting MKL_INCLUDE and MKL_LIBRARY to no avail.

  2. Caffe2 fails to compile because it cannot find a header file for ATen. Here’s the tail of the log:

Build FAILED.

"C:\Users\macolli\Source\pytorch\build\INSTALL.vcxproj" (default target) (1) ->
"C:\Users\macolli\Source\pytorch\build\ALL_BUILD.vcxproj" (default target) (3) ->
"C:\Users\macolli\Source\pytorch\build\caffe2\AlgorithmsTest.vcxproj" (default target) (4) ->
"C:\Users\macolli\Source\pytorch\build\c10\c10.vcxproj" (default target) (5) ->
(ClCompile target) ->
  c:\users\macolli\source\pytorch\c10\c10_dummy.cpp(4): warning C4273: 'c10::HasC10': inconsistent dll linkage [C:\Users\macolli\Source\pytorch\build\c10\c10.vcxproj]


"C:\Users\macolli\Source\pytorch\build\INSTALL.vcxproj" (default target) (1) ->
"C:\Users\macolli\Source\pytorch\build\ALL_BUILD.vcxproj" (default target) (3) ->
"C:\Users\macolli\Source\pytorch\build\caffe2\AlgorithmsTest.vcxproj" (default target) (4) ->
"C:\Users\macolli\Source\pytorch\build\caffe2\caffe2.vcxproj" (default target) (6) ->
"C:\Users\macolli\Source\pytorch\build\caffe2\core\dispatch\dispatch.vcxproj" (default target) (17) ->
(ClCompile target) ->
  c:\users\macolli\source\pytorch\caffe2\core\dispatch\deviceid.h(3): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\core\dispatch\DeviceId.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\core\dispatch\dispatch.vcxproj]
  c:\users\macolli\source\pytorch\caffe2\core\dispatch\deviceid.h(3): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\core\dispatch\DispatchKey.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\core\dispatch\dispatch.vcxproj]
  c:\users\macolli\source\pytorch\caffe2\core\dispatch\layoutid.h(3): fatal error C1083: Cannot open include file: 'ATen/core/IdWrapper.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\core\dispatch\LayoutId.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\core\dispatch\dispatch.vcxproj]
  c:\users\macolli\source\pytorch\caffe2\core\dispatch\deviceid.h(3): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\core\dispatch\KernelRegistration.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\core\dispatch\dispatch.vcxproj]
  c:\users\macolli\source\pytorch\caffe2\core\dispatch\deviceid.h(3): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\core\dispatch\OpSchema.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\core\dispatch\dispatch.vcxproj]
  c:\users\macolli\source\pytorch\caffe2\core\dispatch\deviceid.h(3): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\core\dispatch\Dispatcher.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\core\dispatch\dispatch.vcxproj]
  c:\users\macolli\source\pytorch\caffe2\core\dispatch\deviceid.h(3): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\core\dispatch\DispatchTable.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\core\dispatch\dispatch.vcxproj]
  c:\users\macolli\source\pytorch\caffe2\core\dispatch\deviceid.h(3): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\core\dispatch\OpSchemaRegistration.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\core\dispatch\dispatch.vcxproj]


"C:\Users\macolli\Source\pytorch\build\INSTALL.vcxproj" (default target) (1) ->
"C:\Users\macolli\Source\pytorch\build\ALL_BUILD.vcxproj" (default target) (3) ->
"C:\Users\macolli\Source\pytorch\build\caffe2\utils\c10_utils_cpu.vcxproj" (default target) (40) ->
  c:\users\macolli\source\pytorch\caffe2\utils\array.h(41): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\utils\Array.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\utils\c10_utils_cpu.vcxproj]
  c:\users\macolli\source\pytorch\caffe2\utils\typelist.h(3): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\utils\TypeList.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\utils\c10_utils_cpu.vcxproj]
  c:\users\macolli\source\pytorch\caffe2\utils\typetraits.h(3): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\utils\TypeTraits.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\utils\c10_utils_cpu.vcxproj]
  c:\users\macolli\source\pytorch\caffe2\utils\typelist.h(3): fatal error C1083: Cannot open include file: 'ATen/core/C++17.h': No such file or directory (compiling source file C:\Users\macolli\Source\pytorch\caffe2\utils\Metaprogramming.cpp) [C:\Users\macolli\Source\pytorch\build\caffe2\utils\c10_utils_cpu.vcxproj]

    1 Warning(s)
    12 Error(s)

Time Elapsed 00:05:52.82

This file exists in pytorch/aten/src/ATen/core, but nowhere else.

Thanks,

Marcus

  1. The MKL included in Anaconda is the dynamic version. However, we use the static version during build. And here’s how we enable MKL in CI.
REM Make sure you have 7z and curl installed.

REM Download MKL files
curl https://s3.amazonaws.com/ossci-windows/mkl_2018.2.185.7z -k -O
7z x -aoa mkl_2018.2.185.7z -omkl

REM Setting essential environment variables
set "CMAKE_INCLUDE_PATH=%cd%\\mkl\\include"
set "LIB=%cd%\\mkl\\lib;%LIB%"
  1. Maybe an include dir is missing in the command. Could you please show me the full log?
1 Like

Thanks Peter! It’s picking up MKL now, so that’s progress. Trying to figure out how/where to post the install logs…

@peterjc123 here are the logs: https://gist.github.com/mdc713/679d525e9b2ecdcd1168f2aab675edc8

I couldn’t add a second file without crashing my browser, so the (small) contents of stderr are just in a comment at the bottom. Thank you!

It’s weird since I saw /IC:\Users\macolli\Source\pytorch\aten\src in the command line. Is your clone complete?

It’s totally weird. I verified that the file it is looking for (pytorch/aten/src/ATen/core/C++17.h) is there, and I can open it.