Conflict between libtorch and grpc

C++
1

Hi, I recently tried to use grpc to deploy the model in libtorch (model is just as same as the one in libtorch demo. I had checked grpc installation and it was collectly installed (grpc version: 1.20.0) and when I compiled my project i got an error:

CMakeFiles/resnet_server.dir/resnet_server.cc.o: In function `RunServer()':
resnet_server.cc:(.text+0x11c3): undefined reference to `grpc::ServerBuilder::AddListeningPort(std::string const&, std::shared_ptr<grpc::ServerCredentials>, int*)'
CMakeFiles/resnet_server.dir/__/protos/example.pb.cc.o:(.data.rel.ro._ZTVN7example12DataResponseE[_ZTVN7example12DataResponseE]+0x20): undefined reference to `google::protobuf::Message::GetTypeName() const'
CMakeFiles/resnet_server.dir/__/protos/example.pb.cc.o:(.data.rel.ro._ZTVN7example12DataResponseE[_ZTVN7example12DataResponseE]+0x58): undefined reference to `google::protobuf::Message::InitializationErrorString() const'
CMakeFiles/resnet_server.dir/__/protos/example.pb.cc.o:(.data.rel.ro._ZTVN7example11DataRequestE[_ZTVN7example11DataRequestE]+0x20): undefined reference to `google::protobuf::Message::GetTypeName() const'
CMakeFiles/resnet_server.dir/__/protos/example.pb.cc.o:(.data.rel.ro._ZTVN7example11DataRequestE[_ZTVN7example11DataRequestE]+0x58): undefined reference to `google::protobuf::Message::InitializationErrorString() const'
collect2: error: ld returned 1 exit status
src/CMakeFiles/resnet_server.dir/build.make:156: recipe for target 'src/resnet_server' failed
make[2]: *** [src/resnet_server] Error 1
CMakeFiles/Makefile2:85: recipe for target 'src/CMakeFiles/resnet_server.dir/all' failed
make[1]: *** [src/CMakeFiles/resnet_server.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

I realize that libtorch also uses its own libprotobuf.a libprotobuf-lite.a libprotoc.a, which may have conflict with the corresponding protbuf library of grpc.
Environment:ubuntu 18.04
libtorch: latest cpu version
Can someone help me out and fix this? Thanks

2

There is an issue about it and it’s not fixed yet: https://github.com/pytorch/pytorch/issues/14573. Currently the easiest way is to compile libtorch with the protobuf library that grpc uses, or compile grpc with the protobuf library that libtorch uses.

3

I will try both your suggestions and thank you for your help, I really appreciate it.

4

Any updates? I encountered this problem recently.

5

I faced same issue. I’m new of cpp project.So, I don’t know how to compile libtorch with the protobuf library that grpc uses. Please, I need your help. Can you explain how to compile?

6

You need to manually edit the CMakeLists.txt file such that the line

option(BUILD_CUSTOM_PROTOBUF ... ON)

is changed to

option(BUILD_CUSTOM_PROTOBUF ... OFF)

This will trigger find_package(protobuf) instead of the included protobuf git submodule

Thus you also need to add towards the beginning of the CMakeLists.txt

list(APPEND CMAKE_PREFIX_PATH "{path to your protobuf install root}")

I’m sure there is a less invasive way of achieving the same thing but this works for me.

Note I also had to in command line export LD_LIBRARY_PATH={path to protobuf libs} before compile since the protoc binary is called at least a few times and I built grpc/protobuf using shared libraries.