C10/macros/cmake_macros.h not exists

C++
1

Hi,
I’m trying to include pytorch/script.h in my CPP android project.
I’m using ndk21 and C++14.
I cloned the repository and used “git submodule update --init --recursive” to update sub=modules.
I’m getting the following compilation error:

In file included from C:\REPOS\pytorch\torch/script.h:3:
In file included from C:\REPOS\pytorch\torch/csrc/api/include/torch/types.h:3:
In file included from C:\REPOS\pytorch/aten/src\ATen/ATen.h:3:
In file included from C:\REPOS\pytorch\c10/core/Allocator.h:6:
In file included from C:\REPOS\pytorch\c10/core/Device.h:3:
In file included from C:\REPOS\pytorch\c10/core/DeviceType.h:8:
C:\REPOS\pytorch\c10/macros/Macros.h:21:10: fatal error: ‘c10/macros/cmake_macros.h’ file not found
#include <c10/macros/cmake_macros.h>
^~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

I do see that there is a file c10/macros/cmake_macros.h**.in**, an it specifically says:
Do not include this file directly. Instead, include c10/macros/Macros.h.
So what is the problem?

2

@Aviais
You can’t include a single header file from pytorch codebase directly, there are many dependencies.
Can you tell me the details about what you are trying to do here?

3

  1. I have a CPP android project. I’m compiling it using ndk-build (ndk21 and C++14).

  2. I have a file named “main.cpp” and I’m trying to #include “torch/script.h” .

  3. I got the error above when I git cloned pytorch and updated sub-modules as reported below.

When I downloaded a different libtorch version using this command,
wget https://github.com/Kitware/CMake/releases/download/v3.15.2/cmake-3.15.2.tar.gz
I saw a different structure of the files and I had different compilations error, as I reported in:
Compilation error in 'jit/ir/ir.h'

  1. In my Android.mk file, I’m giving the path to:
    LOCAL_C_INCLUDES += $(TORCH_REPO_PATH)
    $(TORCH_REPO_PATH)/include
    $(TORCH_REPO_PATH)/include/torch
    $(TORCH_REPO_PATH)/include/torch/csrc/api/include

Please help.
Thanks for your response.

4

@Aviais
As I said, you can not include a single header file in your program, it won’t work even you added include path in your mk file, basically you need dependencies, shared libs, at least libc10, libtorch, libtorch_cpu, libtorch_gpu, etc. What I recommend to do is download libtorch zip package and follow the instruction here:
https://pytorch.org/cppdocs/installing.html
And you can extend your makefile based on instruction. Check the CMakeList.txt that provided in the instruction.

If you don’t want to use cmake and want to build with your own makefile or even build with command line, check here:
‘How do i to create a new project in C++ to run libtorch without cmake?’
I posted a command line which you can use to build libtorch. And you can put those minimal requirements in the commandl line (libs, includes, etc) into your make file. But libtorch is changing fast, I do recommend using our cmake solution.

FYI, for torch script, please follow here:
https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html
and this is the c++ part instruction:
https://pytorch.org/tutorials/advanced/cpp_export.html

5

**Thank you for the detailed answer. **
I followed all the instructions you shared with me and did it successfully. I used CMake, built and loaded my model got input and output via CPP on my PC.

I would like to include torch/script.h in my existing CPP Android project, so I can load my model and execute it from my exisiting code.

I added to my Andorid.mk file the shared libraries you mentioned + the includes above.
LOCAL_LDLIBS += -L$(TORCH_REPO_PATH)/lib -ltorch -ltorch_cpu -lc10 -ltorch_global_deps

I’m still getting these errors:

C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:67:17: error: no namespace named ‘prim’ in namespace ‘c10’; did you mean simply ‘prim’?
using namespace ::c10::prim;
^~~~~~~~~~~
prim
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:66:11: note: namespace ‘prim’ defined here
namespace prim {
^
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:70:17: error: no namespace named ‘attr’ in namespace ‘c10’; did you mean simply ‘attr’?
using namespace ::c10::attr;
^~~~~~~~~~~
attr
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:69:11: note: namespace ‘attr’ defined here
namespace attr {
^
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:73:17: error: no namespace named ‘aten’ in namespace ‘c10’; did you mean simply ‘aten’?
using namespace ::c10::aten;
^~~~~~~~~~~
aten
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:72:11: note: namespace ‘aten’ defined here
namespace aten {
^
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:1309:34: error: no member named ‘prim’ in namespace ‘c10’; did you mean simply ‘prim’?
static constexpr Symbol Kind = ::c10::prim::profile;
^~~~~~~~~~~
prim
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:66:11: note: ‘prim’ declared here
namespace prim {
^
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:1309:47: error: no member named ‘profile’ in namespace ‘torch::jit::prim’
static constexpr Symbol Kind = ::c10::prim::profile;
^
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:1311:21: error: no member named ‘prim’ in namespace ‘c10’; did you mean simply ‘prim’?
: Node(graph, ::c10::prim::profile), callback_(callback) {}
^~~~~
prim
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:66:11: note: ‘prim’ declared here
namespace prim {
^
C:\REPOS\CPU\libtorch-shared-with-deps-latest\libtorch/include\torch/csrc/jit/ir/ir.h:1311:34: error: no member named ‘profile’ in namespace ‘torch::jit::prim’
: Node(graph, ::c10::prim::profile), callback_(callback) {}
~~~~~~^
7 errors generated.

What am I missing?

6

@Aviais
What’s your env? Windows? or Linux?

To cross build, you need to make sure the libtorch you downloaded is compiled with the same compiler with your android project. You can download a correct build from here:
‘https://pytorch.org/’ (select package ‘libtorch’, choose correct operating system you have)

If you are under linux and the libtorch you downloaded is non-abi build, then ABI must set to be false.
D_GLIBCXX_USE_CXX11_ABI=0.
And you need to tell linker the location of the shared library. The command line I posted here:
‘How do i to create a new project in C++ to run libtorch without cmake?’
has ‘-Wl,R’ before -ltorch -ltorch_cpu -lc10 so linker knows where to find those shared libs.

7

I did all of the above but I still get the same 7 errors from /include\torch/csrc/jit/ir/ir.h
I’m working on Windows.

8

If you want to compile for Android, then you should not use the Windows libraries. What you are trying to do is called cross compiling, which is to compile for a different target on a host machine. You should choose the libraries that are designed for Android and use GCC instead of MSVC.

9

Thank you for responding.

It is important to mention that I was able to load a torch script model on my PC after I followed the tutorial (using CMake) in: https://pytorch.org/tutorials/advanced/cpp_export.html

Can you please send me a link to a guide that explains how to include pytorch in an existing CPP android project? I tried both linux/windows versions and I get the same 7 errors I reported in this post. I also tried various pytorch versions.
I don’t understand why it doesn’t work…

10

I found a related post. Compilation error in 'jit/ir/ir.h' - #2 by Aviais