The ordinal 242 could not be located in the dynamic link library ***\torch.dll

Valdemar_Erlingsson · February 2, 2020, 4:12pm

Hello. I’m building a low latency, real-time application using a PyTorch model, and I’ve been trying to get a “hello world” kind of example compiling in C++, using either VS2017 or VS2019. So far, I’ve managed to git things to compile, following the tutorial here, but I’m getting a cryptic error at runtime.

The actual message is: "The ordinal 242 could not be located in the dynamic link library c:\dev\libtorch\lib\torch.dll"

Edit: After a lot of work, I have at least been able to verify this only occurs with version 1.4.0 of pytorch, versions 1.3.0 and 1.0.0 both worked fine (I had to guess the url to get download previous releases). I’m currently trying to build pytorch 1.4.0 from source by this is a complex undertaking for someone new to the project.

This happens as soon as I run the compiled executable, regardless of whether I include any code my my main function or not, but as long as my cpp file contains #include <torch/torch.h> this occurs.

If you can help at all, or theorise as to what is going wrong here, I would be very grateful.

Best regards - Valdemar.

More info:

I’ve tried this using both the debug and release versions of libtorch, making sure to have the matching build target when running cmake.
I’ve tried running this with and without the CUDA dlls on my path (various subdirectories from C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1… )
- Edit: I’ve just realized this doesn’t work without the relevant paths to the Cuda libraries, as I get an error:
- – Added CUDA NVCC flags for: -gencode;arch=compute_75,code=sm_75
  CMake Error at C:/dev/libtorch/share/cmake/Caffe2/Caffe2Config.cmake:96 (message):
  Your installed Caffe2 version uses cuDNN but I cannot find the cuDNN
  libraries. Please set the proper cuDNN prefixes and / or install cuDNN.
  Call Stack (most recent call first):
  C:/dev/libtorch/share/cmake/Torch/TorchConfig.cmake:40 (find_package)
  CMakeLists.txt:4 (find_package)
- I’m wondering if I’m mixing up versions of cdnn, and that might be the issue??
I initially had CuDNN 7.6.4.38 installed (latest CuDNN for Cuda 10.1), but have also tried removing that and using 7.6.2.24 instead, which matches the exact dll version of the one in libtorch, but not the size (libtorchj has a 383,443kb file, I have a 417,388kb file)
I’ve tried this with different cmake toolsets, I’ve got both VS2017 and VS2019 installed, and have tried both using, for example cmake -G “Visual Studio 15 2017 Win64” -DCMAKE_PREFIX_PATH=C:\dev\libtorch\
I’ve tried both debug and release version of libtorch.

My environment path contains this at the start:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\libnvvp;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\extras\CUPTI\lib64;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\lib;
C:\dev\libtorch\lib;

I have libtorch located at c:\dev\libtorch

My CMakeLists.txt

cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(dcgan)

find_package(Torch REQUIRED)

add_executable(dcgan dcgan.cpp)
target_link_libraries(dcgan "${TORCH_LIBRARIES}")
set_property(TARGET dcgan PROPERTY CXX_STANDARD 14)

dcgan.cpp

#include <torch/torch.h>
#include <iostream>

int main() {
	//torch::Tensor tensor = torch::eye(3);
	//std::cout << tensor << std::endl;
	std::cout << "hello" << std::endl;
}

My cmake output:

C:\dev\libtorch\test\dcgan>cmake -DCMAKE_PREFIX_PATH=C:\dev\libtorch\
CMake Warning:
  No source or binary directory provided.  Both will be assumed to be the
  same as the current working directory, but note that this warning will
  become a fatal error in future CMake releases.
  
-- Building for: Visual Studio 16 2019
-- The C compiler identification is MSVC 19.23.28107.0
-- The CXX compiler identification is MSVC 19.23.28107.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.23.28105/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.23.28105/bin/Hostx64/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.23.28105/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.23.28105/bin/Hostx64/x64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1 (found version "10.1")
-- Caffe2: CUDA detected: 10.1
-- Caffe2: CUDA nvcc is: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/bin/nvcc.exe
-- Caffe2: CUDA toolkit directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1
-- Caffe2: Header version is: 10.1
-- Found CUDNN: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/lib/x64/cudnn.lib
-- Found cuDNN: v7.6.5  (include: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/include, library: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/lib/x64/cudnn.lib)
-- Autodetected CUDA architecture(s):  7.5
-- Added CUDA NVCC flags for: -gencode;arch=compute_75,code=sm_75
CMake Warning (dev) at C:/dev/libtorch/share/cmake/Caffe2/public/utils.cmake:81 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "MSVC" will no longer be dereferenced when the policy
  is set to NEW.  Since the policy is not set the OLD behavior will be used.
Call Stack (most recent call first):
  C:/dev/libtorch/share/cmake/Caffe2/Caffe2Config.cmake:121 (caffe2_interface_library)
  C:/dev/libtorch/share/cmake/Torch/TorchConfig.cmake:40 (find_package)
  CMakeLists.txt:4 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at C:/dev/libtorch/share/cmake/Torch/TorchConfig.cmake:108 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "MSVC" will no longer be dereferenced when the policy
  is set to NEW.  Since the policy is not set the OLD behavior will be used.
Call Stack (most recent call first):
  CMakeLists.txt:4 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found torch: C:/dev/libtorch/lib/torch.lib
-- Configuring done
-- Generating done
-- Build files have been written to: C:/dev/libtorch/test/dcgan

C:\dev\libtorch\test\dcgan>cmake --build . --config Release
Microsoft (R) Build Engine version 16.3.2+e481bbf88 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

  Checking Build System
  Building Custom Rule C:/dev/libtorch/test/dcgan/CMakeLists.txt
  dcgan.cpp
C:\dev\libtorch\include\torch/csrc/jit/tracer.h(289,72): warning C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\torch/csrc/jit/tracer.h(277,16): message : see previous definition of 'addInputs' [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\torch/csrc/jit/tracer.h(296,44): warning C4273: 'torch::jit::tracer::addInputs': inconsistent dll linkage [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\torch/csrc/jit/tracer.h(283,16): message : see previous definition of 'addInputs' [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/util/Half.h(451,45): warning C4018: '>': signed/unsigned mismatch [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/util/Half.h(497): message : see reference to function template instantiation 'bool c10::overflows<To,From>(From)' being compiled [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/util/Half.h(497): message :         with [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/util/Half.h(497): message :         [ [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/util/Half.h(497): message :             To=int64_t, [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/util/Half.h(497): message :             From=int64_t [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/util/Half.h(497): message :         ] [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/core/Scalar.h(72): message : see reference to function template instantiation 'To c10::checked_convert<int64_t,int64_t>(From,const char *)' being compiled [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/core/Scalar.h(72): message :         with [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/core/Scalar.h(72): message :         [ [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/core/Scalar.h(72): message :             To=int64_t, [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/core/Scalar.h(72): message :             From=int64_t [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
C:\dev\libtorch\include\c10/core/Scalar.h(72): message :         ] [C:\dev\libtorch\test\dcgan\dcgan.vcxproj]
     Creating library C:/dev/libtorch/test/dcgan/Release/dcgan.lib and object C:/dev/libtorch/test/dcgan/Release/dcgan.exp
  dcgan.vcxproj -> C:\dev\libtorch\test\dcgan\Release\dcgan.exe
  Building Custom Rule C:/dev/libtorch/test/dcgan/CMakeLists.txt

C:\dev\libtorch\test\dcgan>

peterjc123 · February 3, 2020, 7:44am

Please add the following code to your cmake scripts:

if (MSVC)
  file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
  add_custom_command(TARGET dcgan
                     POST_BUILD
                     COMMAND ${CMAKE_COMMAND} -E copy_if_different
                     ${TORCH_DLLS}
                     $<TARGET_FILE_DIR:dcgan>)
endif (MSVC)

This code is used to copy all the DLLs from LibTorch to your executable.

Valdemar_Erlingsson · February 3, 2020, 8:04am

Thanks for looking into this.
I’ll try copying the DLLs across but I doubt it will make much of a difference since I’m already including them (directory C:\dev\libtorch\lib) on my path.

Pytorch 1.4.0 seems to ship with a lot more dependencies than previous versions, and I’m wondering what changes between 1.3.0 (which works, and only has a handful of DLLs in the /lib folder) and 1.4.0, which comes with its own version of cudnn.dll and a lot more of the Cuda binaries. I fear something hasn’t been built correctly in the released windows version of 1.4
0

Valdemar_Erlingsson · February 3, 2020, 7:22pm

This is happening with 1.3.1 as well as 1.4.0, but not 1.3.0 or below.
I believe there is some compatibility issue with the binaries that were used to build those versions.

That may, however, be irrelevant, as somehow, when I copy all dlls into the same directory as my built executable, it works. If I don’t copy them in, but instead put the path to those dlls on my PATH (at the front, so it should be the first place it gets picked up!), it does not work and raises the error I described in my original post.

I don’t understand what might be going on there, but I guess I have something else on my path that is causing a DLL hell kind of situation. It is however strange that the older versions work.

For reference, I have Cuda 10.1 and CuDNN 7.6.4.38 installed
My cudart64_101.dll has dll version info 10.1.243 and cudnn64_7.dll is the same

I’ve found a workaround for my issue (for now) by copying the dlls into the same folder as the executable, but I still don’t understand quite what is going on, and would like to, as this is probably going to become an issue once I start deploying this application to our servers

peterjc123 · February 4, 2020, 10:00am

It is simple. paths in PATH are not considered as the first option for finding libraries. see here for more details. https://docs.microsoft.com/zh-cn/windows/win32/dlls/dynamic-link-library-search-order
If you want to see which dll is loaded at runtime, you can refer to the output window of VS or using sth like Process Monitor.