Building libtorch with UE4 - Stuck on a couple of errors

Chozabu · February 22, 2020, 3:32am

Hi all,
First off - Having a Great time with pytorch, and it’s working well with UnrealEngine 4.23 - but with performance in mind…
I’m trying to build libtorch with UE4 - but running into some issues during compilation

Error	C4273	'torch::jit::tracer::addInputs': inconsistent dll linkage	UETorchC	E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\jit\tracer.h	292	
Error	C4273	'torch::jit::tracer::addInputs': inconsistent dll linkage	UETorchC	E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\jit\tracer.h	299	
Error	C2280	'std::shared_ptr<Contained> std::dynamic_pointer_cast<Derived,_Ty>(std::shared_ptr<_Ty> &&) noexcept': attempting to reference a deleted function	UETorchC	E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch\nn\cloneable.h	85

I’m on win10, using VS2019 (which calls UBT - unreals own build system instead of using its own, but still compiles with MSVC)

What I have tried:

building using libtorch 1.4 release, 1.4 debug, and latest debug
surpressing errors using pragma __pragma(warning(disable: 4273))
enabling permissive mode
linking to only the libs, dlls, both or neither (This actually seems to have no effect)
ensuring the build path has no spaces

The only ideas I really have left are to use vs2017 or try building libtorch from source, but not too confident either option will work.

Any ideas? I am happy to provide any more details or share the project.

Thanks!

Maindz · February 22, 2020, 6:12pm

I am trying to get the same thing to work, and have already tried all the things you have tried with no success and I get the same errors you get. In addition I can say that changing to vs2017 will not make a difference as I have tried. Might be a good idea to try to build from source and make sure you follow these steps https://wiki.unrealengine.com/Creating_%26_Linking_Static_Libraries_And_Make_Your_Own_Blueprint_Node_With_VS_2017_%26_UE4, in “Customizations for Targeting UE4 Modules”, I will most likely try that next week. So if you ever figure this one out please let me know.
Thanks.

Chozabu · February 22, 2020, 11:12pm

@Maindz Good to know someone else is on the same path!

Worth asking - what version of UE4 and MSVC are you working with? I’m on MSVC 14.24.28314

Actually - if you can share your build output, could be rather useful to compare - here is mine:

1>------ Build started: Project: UETorchC, Configuration: Development_Editor x64 ------
1>Parsing headers for UETorchCEditor
1>  Running UnrealHeaderTool "E:\UETorchC\UETorchC.uproject" "E:\UETorchC\Intermediate\Build\Win64\UETorchCEditor\Development\UETorchCEditor.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors -installed
1>Reflection code generated for UETorchCEditor in 6.2185253 seconds
1>Using Visual Studio 2019 14.24.28314 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.24.28314) and Windows 10.0.18362.0 SDK (C:\Program Files (x86)\Windows Kits\10).
1>Building 4 actions with 4 processes...
1>  [1/4] UETorch.cpp
1>E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(292): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1>  E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(276): note: see previous definition of 'addInputs'
1>E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(299): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1>  E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(279): note: see previous definition of 'addInputs'
1>E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(85): error C2280: 'std::shared_ptr<Contained> std::dynamic_pointer_cast<Derived,_Ty>(std::shared_ptr<_Ty> &&) noexcept': attempting to reference a deleted function
1>          with
1>          [
1>              Contained=torch::nn::GroupNormImpl,
1>              Derived=torch::nn::GroupNormImpl,
1>              _Ty=torch::nn::Module
1>          ]
1>  C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.24.28314\INCLUDE\memory(1465): note: see declaration of 'std::dynamic_pointer_cast'
1>  E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(81): note: while compiling class template member function 'void torch::nn::Cloneable<torch::nn::GroupNormImpl>::clone_(torch::nn::Module &,const c10::optional<c10::Device> &)'
1>  E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/modules/normalization.h(106): note: see reference to class template instantiation 'torch::nn::Cloneable<torch::nn::GroupNormImpl>' being compiled
1>  E:\UETorchC\Source\ThirdParty\libtorch\include\c10/core/MemoryFormat.h(55): note: see reference to class template instantiation 'c10::ArrayRef<int64_t>' being compiled
1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.MakeFile.Targets(44,5): error MSB3075: The command ""E:\Epic Games\UE_4.23\Engine\Build\BatchFiles\Build.bat" UETorchCEditor Win64 Development -Project="E:\UETorchC\UETorchC.uproject" -WaitMutex -FromMsBuild" exited with code 5. Please verify that you have sufficient rights to run this command.
1>Done building project "UETorchC.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

strongly suspect issues are MSVC version related, I’ll give this a go on my linux box - avoiding the MSVC part of the toolchain may well sidestep the issues…

Maindz · February 23, 2020, 1:32pm

I have tried different MSVC versions but the one I am using now is 2017 14.16.27023.
Build output:

1>------ Build started: Project: IAF, Configuration: Development_Editor x64 ------
1>Using 'git status' to determine working set for adaptive non-unity build (C:\Users\mariu\Documents\Bachelor\IAF).
1>Invalidating makefile for IAFEditor (source file removed)
1>Parsing headers for IAFEditor
1>  Running UnrealHeaderTool "C:\Users\mariu\Documents\Bachelor\IAF\IAF.uproject" "C:\Users\mariu\Documents\Bachelor\IAF\Intermediate\Build\Win64\IAFEditor\Development\IAFEditor.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors -installed
1>Reflection code generated for IAFEditor in 6,4612741 seconds
1>Using Visual Studio 2017 14.16.27023 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023) and Windows 10.0.17763.0 SDK (C:\Program Files (x86)\Windows Kits\10).
1>Building 12 actions with 8 processes...
1>  [1/12] SharedPCH.Engine.NoUndef.cpp
1>  [2/12] IAF.cpp
1>  [3/12] RL.cpp
1>  [4/12] PPOActor.cpp
1>  [5/12] IAF.init.gen.cpp
1>  [6/12] AgentPPO.gen.cpp
1>  [7/12] RL.gen.cpp
1>  [8/12] PPOActor.gen.cpp
1>  [9/12] AgentPPO.cpp
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/flat_hash_map.h(1308): warning C4458: declaration of 'shift' hides class member
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/flat_hash_map.h(1318): note: see declaration of 'ska::fibonacci_hash_policy::shift'
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\ATen/core/TensorBody.h(1105): error C4522: 'at::Tensor': multiple assignment operators specified
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/order_preserving_flat_hash_map.h(1427): warning C4458: declaration of 'shift' hides class member
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/order_preserving_flat_hash_map.h(1437): note: see declaration of 'ska_ordered::fibonacci_hash_policy::shift'
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(289): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(277): note: see previous definition of 'addInputs'
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(296): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(283): note: see previous definition of 'addInputs'
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(85): error C2280: 'std::shared_ptr<Contained> std::dynamic_pointer_cast<Derived,_Ty>(const std::shared_ptr<_Ty> &) noexcept': attempting to reference a deleted function
1>          with
1>          [
1>              Contained=torch::nn::GroupNormImpl,
1>              Derived=torch::nn::GroupNormImpl,
1>              _Ty=torch::nn::Module
1>          ]
1>  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\INCLUDE\memory(1770): note: see declaration of 'std::dynamic_pointer_cast'
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(81): note: while compiling class template member function 'void torch::nn::Cloneable<torch::nn::GroupNormImpl>::clone_(torch::nn::Module &,const c10::optional<c10::Device> &)'
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/modules/normalization.h(106): note: see reference to class template instantiation 'torch::nn::Cloneable<torch::nn::GroupNormImpl>' being compiled
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/core/MemoryFormat.h(53): note: see reference to class template instantiation 'c10::ArrayRef<int64_t>' being compiled
1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.MakeFile.Targets(44,5): error MSB3075: The command ""C:\Program Files\Epic Games\UE_4.23\Engine\Build\BatchFiles\Build.bat" IAFEditor Win64 Development -Project="C:\Users\mariu\Documents\Bachelor\IAF\IAF.uproject" -WaitMutex -FromMsBuild" exited with code 5. Please verify that you have sufficient rights to run this command.
1>Done building project "IAF.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

AgentPPO is the first place I include libtorch.

If I remember correctly I think the c4458 warnings and the c4522 error disappeared when I changed to libtorch gpu version, used libtorch 1.4 cpu when I tried to build this, though I have tried pretty much every version from 1.2 both cpu and gpu.

Probably also worth mentioning that I tried to use libtorch in visual studio without UE4 just a plain c++ project, and that worked.

Selcuk_Caglar · March 25, 2020, 9:05am

Have got any progress? What is the problem with GPU version?

Chozabu · April 5, 2020, 1:30pm

I did try compilation from linux - and got a different error, seemed more understandable but not yet looked into it deeply.

I’ve been working more with the regular python version of pytorch + ue4 for now: https://gyazo.com/5adf2471d03be02aefe4a5a1ea4d5a3e

Still intending to revisit compiling libtorch with UE4, but not sure when.
Also considering other options - like a lightweight inference system - similar to barracuda

David_Yanni · April 23, 2020, 7:46pm

I got help for this from someone at work who is a ue4 pro. Try enabling RTTI.

Edit also the warnings can be gotten around with

THIRD_PARTY_INCLUDES_START
#include "torch/torch.h"
THIRD_PARTY_INCLUDES_END

Chozabu · April 27, 2020, 9:37pm

@David_Yanni Interesting! Am I to understand you have libtorch up and running with ue4?

Loris_Donovan · May 10, 2020, 1:42pm

So after following @David_Yanni’s steps and adding a few changes in the definition of THIRD_PARTY_INCLUDES_START i.e, after I adding the following lines in the definition:

__pragma(warning(disable: 4273))\
__pragma(warning(disable: 4582))\
__pragma(warning(disable: 4583))\
__pragma(warning(disable: 4018))\

And also copying all the .dll files in libtorch/lib folder to Binaries/Win64, I managed to get the project compiling. But I don’t seem to be able to “Play” after adding blueprint class which simply logs the output of a neural network. I get a breakpoint saying “ntdll.pdb not loaded”.
Since I am new to Unreal Engine may be I did something wrong idk so I would like you to try it out. If you get it working please let me know. Thanks!

Fhrozen · May 18, 2020, 10:35am

Hello there, interesting project going here.
I just finished to test an integration btw Unity and Libtorch following this recipe: https://www.goodai.com/neural-networks-in-unity-using-native-libraries/.
Then, I would suggest to build an external library similar to the link and then just import the built library into UE4. I would try this also for a project later.
Let me know if it works.

Best

NeuralVFX · July 30, 2020, 11:28am

@Fhrozen Did you ever happen to get Unreal+Librorch working? I just followed this “goodai.com” tutorial you posted, except using Unreal Engine instead of Unity, and it works. My test scene runs salient object detection on a live video feed, passes the output through the DLL, and updates a texture in the scene every tick.

I don’t have a GPU at my disposal… so I used “libtorch-win-shared-with-deps-1.5.1+cpu.zip”

I’m curious if you or anyone here can confirm if the same thing works with the GPU version.

Fhrozen · July 31, 2020, 12:14am

I have no problem with the GPU version.
In my case, tested on Unity, I added the line

network.to(at::kCUDA); // If we're doing this on GPU

in the network.cpp file and it worked. I supposed that this should work for UE4 too

ava6969 · August 9, 2020, 7:54pm

Hi Nelson where you able to get unreal working with libtorch. my goal is to be able to use the libtorch library seamlessly with unreal projects, rather than a network.

ava6969 · August 9, 2020, 7:55pm

Hi loris where you able to get unreal working with libtorch. my goal is to be able to use the libtorch library seamlessly with unreal projects, rather than a network.

ava6969 · August 9, 2020, 7:55pm

Hi david where you able to get unreal working with libtorch. my goal is to be able to use the libtorch library seamlessly with unreal projects, rather than a network.

Fhrozen · August 10, 2020, 11:55pm

I did not try to run with unreal and libtorch seamlessly. I only tested on Unity. Sadly, I do not have time for test it.

ava6969 · August 11, 2020, 6:27am

Can you possibly share a git link also for your unity project. if its public. i had a hard time following the instruction.

Fhrozen · August 17, 2020, 11:49pm

I did not have an open git for that project, but I copied from here https://www.goodai.com/neural-networks-in-unity-using-native-libraries/. I will try to upload the code later.

NeuralVFX · August 24, 2020, 8:11pm

In case anyone’s interested, I’ve made an example UE4 project based on the goodai tutorial:

DLL Project: https://github.com/NeuralVFX/basic-libtorch-dll

Unreal Project: https://github.com/NeuralVFX/basic-unreal-libtorch-plugin

The example uses the CPU, however, if @Fhrozen is right, changing a couple lines of code will make it GPU compatible.

xindelvcheng · February 10, 2021, 5:57am

This problem can be solve by add "bUseRTTI = true;"to *.build.cs。I dumpbin *.dll (to search dependency)and link successfully，but it work abnormally，every tensor is filled by 0。