Hello,
I have written a custom kernel for CPUs with the torch C++ extension and I want to compile it just-in-time, using torch.utils.cpp_extension.load. The compilation process works well on a Linux machine, but it fails on MacOS. In particular, the header file TargetConditionals.h seems to be missing.
Here is a excerpt of the logs during the compilation process:
RuntimeError: Error building extension 'test': [1/2] /Users/boh/opt/anaconda3/envs/TestEnv/bin/clang -MMD -MF test.o.d -DTORCH_EXTENSION_NAME=test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include -isystem /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/TH -isystem /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/THC -isystem /Users/boh/opt/anaconda3/envs/TestEnv/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -fopenmp -c /Users/boh/git/TestEnv/test.cpp -o test.o
FAILED: test.o
/Users/boh/opt/anaconda3/envs/TestEnv/bin/clang -MMD -MF test.o.d -DTORCH_EXTENSION_NAME=test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include -isystem /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/TH -isystem /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/THC -isystem /Users/boh/opt/anaconda3/envs/TestEnv/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -fopenmp -c /Users/boh/git/TestEnv/test.cpp -o test.o
In file included from /Users/boh/git/TestEnv/test.cpp:1:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/torch/extension.h:4:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:7:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/autograd.h:3:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/torch/csrc/autograd/autograd.h:3:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/torch/csrc/autograd/variable.h:6:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/torch/csrc/autograd/cpp_hook.h:2:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/torch/csrc/autograd/function_hook.h:3:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/ATen/Tensor.h:3:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:11:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/c10/core/Device.h:3:
In file included from /Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/c10/core/DeviceType.h:8:
/Users/boh/opt/anaconda3/envs/TestEnv/lib/python3.10/site-packages/torch/include/c10/macros/Macros.h:414:10: fatal error: 'TargetConditionals.h' file not found
#include <TargetConditionals.h>
^~~~~~~~~~~~~~~~~~~~~~
1 error generated.
ninja: build stopped: subcommand failed.
Has anybody experienced a similar issue and could help me resolve this?