Generating and Calling custom C++ LibTorch Functions in Python script via SetupTools/CMakesLists.txt

1

In General, I need a shared library file generated (.so) which I can load using <torch.ops.load_library()> function from a python script.

Working on Windows 10: 64 bit-operating system

Tried 2 approaches- Neither generate this File

1> CMakesList.txt - The build is succesful and I get a .dll and .lib file which I tried loading using python ctypes - cdll.load_library(), but no functions present, cross checked using dumpbin via vs code.

cmake_minimum_required (VERSION 3.8)
    find_package(Torch REQUIRED)
    project(reductionResNet VERSION 1.0 DESCRIPTION "Deep_Learning")
    
    # add_executable (customOperator "customOperator.cpp" "customOperator.h")
    # Define our library target
    add_library(customOperator SHARED customOperator.cpp)
    # Enable C++14
    target_compile_features(customOperator PRIVATE cxx_std_14)
    # Link against LibTorch
    target_link_libraries(customOperator "${TORCH_LIBRARIES}")
    
    if (MSVC)
    	file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
    	add_custom_command(TARGET customOperator
    						POST_BUILD
    						COMMAND ${CMAKE_COMMAND} -E copy_if_different
    						${TORCH_DLLS}
    						$<TARGET_FILE_DIR:customOperator>)
    endif (MSVC)

2> Python Setuptools.py - Throws the error → LINK : error LNK2001: unresolved external symbol PyInit_reduction.

 from setuptools import setup, Extension
    from torch.utils import cpp_extension
    
    setup(name='customOperator',
          ext_modules=[cpp_extension.CppExtension('customOperator', ['customOperator.cpp'],
          include_dirs = ['C:/Libtorch/libtorch-win-shared-with-deps-1.8.1+cpu/libtorch'])],
          cmdclass={'build_ext':cpp_extension.BuildExtension.with_options(no_python_abi_suffix=True)}  )

These are the tutorials I have been following pytorchDocs & gitTutorial

This is the Tested .cpp file that holds the two functions - reduction and repeatInterleave

 #include "customOperator.h"
    #include <torch/torch.h>
    
    using namespace std;
    
    torch::Tensor repeatInterleave(
        torch::Tensor input,
        int val
    ) {
        auto output_ = torch::repeat_interleave(input, val);
        return output_;
    }
    
    torch::Tensor reduction(
        torch::Tensor layerOne,
        torch::Tensor layerTwo,
        torch::Tensor layerThree,
        torch::Tensor layerFour) {
    
    
        auto layerOne_ = repeatInterleave(layerOne, 8);
        auto layerTwo_ = repeatInterleave(layerTwo, 4);
        auto layerThree_ = repeatInterleave(layerThree, 2);
    
        int len = layerFour.sizes()[0];
        //cout << len << endl;
    
        torch::Tensor arr[512] = {};
    
        //torch::Tensor* arr = new torch::Tensor[len];
        //std::vector<std::string> x = { "a", "b", "c" };
        //x.push_back("d");
        //std::vector <torch::Tensor> arr = {};
    
        for (int i = 0; i < len; i += 1) {
            arr[i] = (layerOne_[i] + layerTwo_[i] + layerThree_[i] + layerFour[i]) / 4;
    
            //arr.push_back((layerOne_[i] + layerTwo_[i] + layerThree_[i] + layerFour[i]) / 4);
            //cout << arr[i] << endl;
        }
    
        //cout << arr << endl;
        //torch::Tensor output = torch::zeros(layerFour.sizes());
        //delete[] arr;
    
        auto ouput = torch::stack(arr);
        return ouput;
    }
    
    int main()
    {
    	cout << "Hello CMake." << endl;
    	return 0;
    }
2

There are several methods to achieve this, but I think you need to decide on one and then add the required interface declarations. (I think it’s a DLL on Python, not a .so btw.)

  • If you want to do C++ and Python, the custom operator approach you mention is a nice way.
    For this, too you would want to add a the registration part, this can look like the one in the custom op tutorial:

    TORCH_LIBRARY(my_ops, m) {
   m.def("warp_perspective", warp_perspective);
}

    This can be loaded with torch.ops.load_library.

  • If you want to call from Python: Write a torch extension module.
    In this case you need a declaration like this one from the tutorial:

    PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
 m.def("forward", &lltm_forward, "LLTM forward");
 m.def("backward", &lltm_backward, "LLTM backward");
}

    This seems to be missing in method 2 and the error messages is a cryptic way of telling you “this isn’t a complete python module”.

As an aside:

  • If you want to call from C++ only: you can skip the PYBIND11_MODULE likely need to declare your symbols as exported. This is particularly important on windows. (__declspec(dllexport) in the exporting library, __declspec(dllimport) in the programs using it). This seems to be the problem of of no symbols find in method 1. You don’t need a main for libraries (I think). I’m labelling this “C++ only” because I don’t know how well passing tensors via ctypes would work. You could try to go through DLPack if you wanted. I do use DLPack to get tensors to get third party C code (e.g. Apache TVM-generated kernels) in my courses, but even there I do the conversion in a custom PyTorch C++ extension module.

Best regards

Thomas