When I run the following code which is a simple test of cpp code in python, I get errors from Ninja and cl.exe.
Blockquote
import os
import torch
from torch.utils.cpp_extension import load_inline
cpp_source = “”"
#include <torch/extension.h>
torch::Tensor add_one(torch::Tensor input) {
return input + 1;
}
PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
m.def(“add_one”, &add_one, “Add one to each element of the tensor”);
}
“”"
build_dir = “G:\test-python\HSH-Challenge\MyTests-Pytorch”
os.environ[“VS_PATH”] = “C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.21.27702\bin\Hostx64\x64”
module = load_inline(
name=‘add_one_extension’,
cpp_sources=cpp_source,
functions=[‘add_one’],
verbose=True,
build_directory=build_dir
)
tensor = torch.tensor([1, 2, 3])
print(module.add_one(tensor))
The error is :
Blockquote
Emitting ninja build file G:\test-python\HSH-Challenge\MyTests-Pytorch\build.ninja…
Building extension module add_one_extension…
Allowing ninja to set a default number of workers… (overridable by setting the environment variable MAX_JOBS=N)
[1/2] cl /showIncludes -DTORCH_EXTENSION_NAME=add_one_extension -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\include -IC:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\include\TH -IC:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\include\THC -IC:\ProgramData\anaconda3\envs\diffrl\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c G:\test-python\HSH-Challenge\MyTests-Pytorch\main.cpp /Fomain.o
FAILED: main.o
cl /showIncludes -DTORCH_EXTENSION_NAME=add_one_extension -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\include -IC:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\include\TH -IC:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\include\THC -IC:\ProgramData\anaconda3\envs\diffrl\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c G:\test-python\HSH-Challenge\MyTests-Pytorch\main.cpp /Fomain.o
Microsoft (R) C/C++ Optimizing Compiler Version 19.21.27702.2 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
C:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\include\c10/macros/Macros.h(3): fatal error C1083: Cannot open include file: ‘cassert’: No such file or directory
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File “C:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\utils\cpp_extension.py”, line 1900, in _run_ninja_build
subprocess.run(
File “C:\ProgramData\anaconda3\envs\diffrl\lib\subprocess.py”, line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command ‘[‘ninja’, ‘-v’]’ returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “g:\test-python\HSH-Challenge\MyTests-Pytorch\my_torch.py”, line 22, in
module = load_inline(
File “C:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\utils\cpp_extension.py”, line 1433, in load_inline
return _jit_compile(
File “C:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\utils\cpp_extension.py”, line 1508, in _jit_compile
_write_ninja_file_and_build_library(
File “C:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\utils\cpp_extension.py”, line 1623, in _write_ninja_file_and_build_library
_run_ninja_build(
File “C:\ProgramData\anaconda3\envs\diffrl\lib\site-packages\torch\utils\cpp_extension.py”, line 1916, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension ‘add_one_extension’
It seems that the problem is about cl.exe but I have no idea about it. My Visual Studio version is 2019, Pytorch version is 1.13.1 and Cuda version is 11.7.
Also Ninja has built main.cpp and build.ninja files. Thanks for your help