Building libtorch with UE4 - Stuck on a couple of errors

Hi all,
First off - Having a Great time with pytorch, and it’s working well with UnrealEngine 4.23 - but with performance in mind…
I’m trying to build libtorch with UE4 - but running into some issues during compilation

Error	C4273	'torch::jit::tracer::addInputs': inconsistent dll linkage	UETorchC	E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\jit\tracer.h	292	
Error	C4273	'torch::jit::tracer::addInputs': inconsistent dll linkage	UETorchC	E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\jit\tracer.h	299	
Error	C2280	'std::shared_ptr<Contained> std::dynamic_pointer_cast<Derived,_Ty>(std::shared_ptr<_Ty> &&) noexcept': attempting to reference a deleted function	UETorchC	E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch\nn\cloneable.h	85	

I’m on win10, using VS2019 (which calls UBT - unreals own build system instead of using its own, but still compiles with MSVC)

What I have tried:

  • building using libtorch 1.4 release, 1.4 debug, and latest debug
  • surpressing errors using pragma __pragma(warning(disable: 4273))
  • enabling permissive mode
  • linking to only the libs, dlls, both or neither (This actually seems to have no effect)
  • ensuring the build path has no spaces

The only ideas I really have left are to use vs2017 or try building libtorch from source, but not too confident either option will work.

Any ideas? I am happy to provide any more details or share the project.

Thanks!

2 Likes

I am trying to get the same thing to work, and have already tried all the things you have tried with no success and I get the same errors you get. In addition I can say that changing to vs2017 will not make a difference as I have tried. Might be a good idea to try to build from source and make sure you follow these steps https://wiki.unrealengine.com/Creating_%26_Linking_Static_Libraries_And_Make_Your_Own_Blueprint_Node_With_VS_2017_%26_UE4, in “Customizations for Targeting UE4 Modules”, I will most likely try that next week. So if you ever figure this one out please let me know.
Thanks.

@Maindz Good to know someone else is on the same path!

Worth asking - what version of UE4 and MSVC are you working with? I’m on MSVC 14.24.28314

Actually - if you can share your build output, could be rather useful to compare - here is mine:

1>------ Build started: Project: UETorchC, Configuration: Development_Editor x64 ------
1>Parsing headers for UETorchCEditor
1>  Running UnrealHeaderTool "E:\UETorchC\UETorchC.uproject" "E:\UETorchC\Intermediate\Build\Win64\UETorchCEditor\Development\UETorchCEditor.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors -installed
1>Reflection code generated for UETorchCEditor in 6.2185253 seconds
1>Using Visual Studio 2019 14.24.28314 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.24.28314) and Windows 10.0.18362.0 SDK (C:\Program Files (x86)\Windows Kits\10).
1>Building 4 actions with 4 processes...
1>  [1/4] UETorch.cpp
1>E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(292): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1>  E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(276): note: see previous definition of 'addInputs'
1>E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(299): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1>  E:\UETorchC\Source\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(279): note: see previous definition of 'addInputs'
1>E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(85): error C2280: 'std::shared_ptr<Contained> std::dynamic_pointer_cast<Derived,_Ty>(std::shared_ptr<_Ty> &&) noexcept': attempting to reference a deleted function
1>          with
1>          [
1>              Contained=torch::nn::GroupNormImpl,
1>              Derived=torch::nn::GroupNormImpl,
1>              _Ty=torch::nn::Module
1>          ]
1>  C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.24.28314\INCLUDE\memory(1465): note: see declaration of 'std::dynamic_pointer_cast'
1>  E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(81): note: while compiling class template member function 'void torch::nn::Cloneable<torch::nn::GroupNormImpl>::clone_(torch::nn::Module &,const c10::optional<c10::Device> &)'
1>  E:\UETorchC\Source\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/modules/normalization.h(106): note: see reference to class template instantiation 'torch::nn::Cloneable<torch::nn::GroupNormImpl>' being compiled
1>  E:\UETorchC\Source\ThirdParty\libtorch\include\c10/core/MemoryFormat.h(55): note: see reference to class template instantiation 'c10::ArrayRef<int64_t>' being compiled
1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.MakeFile.Targets(44,5): error MSB3075: The command ""E:\Epic Games\UE_4.23\Engine\Build\BatchFiles\Build.bat" UETorchCEditor Win64 Development -Project="E:\UETorchC\UETorchC.uproject" -WaitMutex -FromMsBuild" exited with code 5. Please verify that you have sufficient rights to run this command.
1>Done building project "UETorchC.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

strongly suspect issues are MSVC version related, I’ll give this a go on my linux box - avoiding the MSVC part of the toolchain may well sidestep the issues…

I have tried different MSVC versions but the one I am using now is 2017 14.16.27023.
Build output:

1>------ Build started: Project: IAF, Configuration: Development_Editor x64 ------
1>Using 'git status' to determine working set for adaptive non-unity build (C:\Users\mariu\Documents\Bachelor\IAF).
1>Invalidating makefile for IAFEditor (source file removed)
1>Parsing headers for IAFEditor
1>  Running UnrealHeaderTool "C:\Users\mariu\Documents\Bachelor\IAF\IAF.uproject" "C:\Users\mariu\Documents\Bachelor\IAF\Intermediate\Build\Win64\IAFEditor\Development\IAFEditor.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors -installed
1>Reflection code generated for IAFEditor in 6,4612741 seconds
1>Using Visual Studio 2017 14.16.27023 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023) and Windows 10.0.17763.0 SDK (C:\Program Files (x86)\Windows Kits\10).
1>Building 12 actions with 8 processes...
1>  [1/12] SharedPCH.Engine.NoUndef.cpp
1>  [2/12] IAF.cpp
1>  [3/12] RL.cpp
1>  [4/12] PPOActor.cpp
1>  [5/12] IAF.init.gen.cpp
1>  [6/12] AgentPPO.gen.cpp
1>  [7/12] RL.gen.cpp
1>  [8/12] PPOActor.gen.cpp
1>  [9/12] AgentPPO.cpp
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/flat_hash_map.h(1308): warning C4458: declaration of 'shift' hides class member
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/flat_hash_map.h(1318): note: see declaration of 'ska::fibonacci_hash_policy::shift'
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\ATen/core/TensorBody.h(1105): error C4522: 'at::Tensor': multiple assignment operators specified
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/order_preserving_flat_hash_map.h(1427): warning C4458: declaration of 'shift' hides class member
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/util/order_preserving_flat_hash_map.h(1437): note: see declaration of 'ska_ordered::fibonacci_hash_policy::shift'
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(289): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(277): note: see previous definition of 'addInputs'
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(296): error C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch/csrc/jit/tracer.h(283): note: see previous definition of 'addInputs'
1>C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(85): error C2280: 'std::shared_ptr<Contained> std::dynamic_pointer_cast<Derived,_Ty>(const std::shared_ptr<_Ty> &) noexcept': attempting to reference a deleted function
1>          with
1>          [
1>              Contained=torch::nn::GroupNormImpl,
1>              Derived=torch::nn::GroupNormImpl,
1>              _Ty=torch::nn::Module
1>          ]
1>  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\INCLUDE\memory(1770): note: see declaration of 'std::dynamic_pointer_cast'
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/cloneable.h(81): note: while compiling class template member function 'void torch::nn::Cloneable<torch::nn::GroupNormImpl>::clone_(torch::nn::Module &,const c10::optional<c10::Device> &)'
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\torch\csrc\api\include\torch/nn/modules/normalization.h(106): note: see reference to class template instantiation 'torch::nn::Cloneable<torch::nn::GroupNormImpl>' being compiled
1>  C:\Users\mariu\Documents\Bachelor\IAF\ThirdParty\libtorch\include\c10/core/MemoryFormat.h(53): note: see reference to class template instantiation 'c10::ArrayRef<int64_t>' being compiled
1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.MakeFile.Targets(44,5): error MSB3075: The command ""C:\Program Files\Epic Games\UE_4.23\Engine\Build\BatchFiles\Build.bat" IAFEditor Win64 Development -Project="C:\Users\mariu\Documents\Bachelor\IAF\IAF.uproject" -WaitMutex -FromMsBuild" exited with code 5. Please verify that you have sufficient rights to run this command.
1>Done building project "IAF.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

AgentPPO is the first place I include libtorch.

If I remember correctly I think the c4458 warnings and the c4522 error disappeared when I changed to libtorch gpu version, used libtorch 1.4 cpu when I tried to build this, though I have tried pretty much every version from 1.2 both cpu and gpu.

Probably also worth mentioning that I tried to use libtorch in visual studio without UE4 just a plain c++ project, and that worked.

Have got any progress? What is the problem with GPU version?

I did try compilation from linux - and got a different error, seemed more understandable but not yet looked into it deeply.

I’ve been working more with the regular python version of pytorch + ue4 for now: https://gyazo.com/5adf2471d03be02aefe4a5a1ea4d5a3e

Still intending to revisit compiling libtorch with UE4, but not sure when.
Also considering other options - like a lightweight inference system - similar to barracuda

I got help for this from someone at work who is a ue4 pro. Try enabling RTTI.

Edit also the warnings can be gotten around with

THIRD_PARTY_INCLUDES_START
#include "torch/torch.h"
THIRD_PARTY_INCLUDES_END

@David_Yanni Interesting! Am I to understand you have libtorch up and running with ue4?

So after following @David_Yanni’s steps and adding a few changes in the definition of THIRD_PARTY_INCLUDES_START i.e, after I adding the following lines in the definition:

__pragma(warning(disable: 4273))\
__pragma(warning(disable: 4582))\
__pragma(warning(disable: 4583))\
__pragma(warning(disable: 4018))\

And also copying all the .dll files in libtorch/lib folder to Binaries/Win64, I managed to get the project compiling. But I don’t seem to be able to “Play” after adding blueprint class which simply logs the output of a neural network. I get a breakpoint saying “ntdll.pdb not loaded”.
Since I am new to Unreal Engine may be I did something wrong idk so I would like you to try it out. If you get it working please let me know. Thanks!

Hello there, interesting project going here.
I just finished to test an integration btw Unity and Libtorch following this recipe: https://www.goodai.com/neural-networks-in-unity-using-native-libraries/.
Then, I would suggest to build an external library similar to the link and then just import the built library into UE4. I would try this also for a project later.
Let me know if it works.

Best

2 Likes

@Fhrozen Did you ever happen to get Unreal+Librorch working? I just followed this “goodai.com” tutorial you posted, except using Unreal Engine instead of Unity, and it works. My test scene runs salient object detection on a live video feed, passes the output through the DLL, and updates a texture in the scene every tick.

I don’t have a GPU at my disposal… so I used “libtorch-win-shared-with-deps-1.5.1+cpu.zip”

I’m curious if you or anyone here can confirm if the same thing works with the GPU version.

I have no problem with the GPU version.
In my case, tested on Unity, I added the line

network.to(at::kCUDA); // If we're doing this on GPU

in the network.cpp file and it worked. I supposed that this should work for UE4 too

Hi Nelson where you able to get unreal working with libtorch. my goal is to be able to use the libtorch library seamlessly with unreal projects, rather than a network.

Hi loris where you able to get unreal working with libtorch. my goal is to be able to use the libtorch library seamlessly with unreal projects, rather than a network.

Hi david where you able to get unreal working with libtorch. my goal is to be able to use the libtorch library seamlessly with unreal projects, rather than a network.

I did not try to run with unreal and libtorch seamlessly. I only tested on Unity. Sadly, I do not have time for test it.

Can you possibly share a git link also for your unity project. if its public. i had a hard time following the instruction.

I did not have an open git for that project, but I copied from here https://www.goodai.com/neural-networks-in-unity-using-native-libraries/. I will try to upload the code later.

In case anyone’s interested, I’ve made an example UE4 project based on the goodai tutorial:

DLL Project: https://github.com/NeuralVFX/basic-libtorch-dll

Unreal Project: https://github.com/NeuralVFX/basic-unreal-libtorch-plugin

The example uses the CPU, however, if @Fhrozen is right, changing a couple lines of code will make it GPU compatible.

This problem can be solve by add "bUseRTTI = true;"to *.build.cs。I dumpbin *.dll (to search dependency)and link successfully,but it work abnormally,every tensor is filled by 0。