Unable to read Data using PyTorch C++ API (MNIST)

C++
1

I’m trying to build MNIST sample file from PyTorch Examples. Here’s what I’ve followed as of now:

  1. Clone examples.
  2. Open x64 Native tools Command prompt for VS 2019
  3. Go to the directory and cmake -DCMAKE_PREFIX_PATH="C:\Users\krshr\Downloads\libtorch\libtorch" -G "NMake Makefiles"
  4. The files were generated successfully.
  5. I then ran “nmake”
  6. It shows some warnings but at last it shows: “Build target mnist”.
  7. I then copied the dll files (torch.dll, caffe2.dll and c10.dll) from unzipped libtorch (nightly version) to the directory having executable file.
  8. Once done, I ran: “mnist.exe” in the prompt and looks like it wasn’t successful while loading MNIST data (I had 2 cout statements before and after loading data - only before one gets printed to the console). The error is: abort() has been called

For CMakeLists.txt and Code: https://github.com/krshrimali/MNIST-Libtorch-Windows

Versions: CMake: 3.15.0-rc1 and Visual Studio 2019 Community

2

You need all these DLL’s inside the folder where mnist.exe is:

file(COPY "${DLL_3RD}/torch.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/hdf5.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/opencv_world400.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/zlib.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/caffe2_gpu.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/nvToolsExt64_1.dll" DESTINATION ${CMAKE_BINARY_DIR})
file(COPY "${DLL_3RD}/c10_cuda.dll" DESTINATION ${CMAKE_BINARY_DIR})

This is for the CUDA version.

3

Also, do you have this in your Cmake?

set(Torch_DIR f:/libtorch_gpu/share/cmake/Torch/)
FIND_PACKAGE(Torch REQUIRED)

find_package(CUDA REQUIRED)
if(CUDA_FOUND)
    # There is some bug in CMake (FindCUDA.cmake) that causes it to mess up the -std=c++11 option
    # if you let it propagate it to nvcc in some cases.
    if(NOT MSVC)
        set(CUDA_PROPAGATE_HOST_FLAGS OFF)
        list(APPEND CUDA_NVCC_FLAGS -std=c++11)
    endif()
endif()
4

Thanks for your response @dambo. I am trying to run non-CUDA version of libtorch. For the DLL files you mentioned, looks like most of the DLL files are for CUDA version.

As far as the CMake is concerned, I added Torch_DIR and CMAKE_PREFIX_PATH to CMakeLists.txt. It builds successfully as before, but exits abruptly without showing any error or message. [Also, the problem is with the data loading as the cout statement after data loader doesn’t get executed]

I’ve updated the files I’ve here: https://github.com/krshrimali/MNIST-Libtorch-Windows

Thanks! Will appreciate any help.

5

I found this warning message after nmake command.

image
image1351×227 13.6 KB

Not sure if this has to do with the above problem.

6

Do the following:
Right click your executable like so:
2019-06-14%2018_25_33-
The run it and wait for windows to report any missing DLL’s.

7

I tried doing that, it says that it made it compatible for Windows 8 mode and asked to test again. The test fails again.

8

What is the exception?

9

I troubleshooted and it then showed the following:

image

When I test the program, the executable file ends just before reading data (without showing any error/exception).

10

I tried to use cmake -g "Ninja" command and then did ninja to make. After running mnist.exe here is the assertion failed error. After ignoring, it ends immediately.

image
image882×496 14.9 KB

11

Add this to the cmake command -DCMAKE_BUILD_TYPE=Release. The debug and release binaries are not ABI compatible when you use MSVC.

12

Thanks for the response @peterjc123. I tried doing that, but it still stops abruptly without showing any exception or error.

I also tried downloading the Stable Version of libtorch again, but it still doesn’t work.

13

Just wondering if the warnings during the make command make any sense? If there’s anything wrong?

image
image1349×454 21.4 KB

14

Please build with the debugging version of this library and use VS GUI with the debug configuration to get more info.

15

And you can add /EHsc to the compiler flags so that the exceptions could be raised.

16

Thanks for the suggestions, @peterjc123. Here is the error I get while trying to run the executable file.

image
image971×504 26.6 KB

By VS GUI, did you mean to try building with Microsoft Visual Studio and not with Ninja?

17

Yes. I mean IDE. You’ll need to add Include Directories and Linking libraries manually.

18

I right clicked on the exception and clicked on “Go to source code”:

image
image1366×727 77.2 KB

I then went to the included path of libtorch (debug version) and couldn’t find mnist.cpp in the required location. Is this a mistake?

image
image1366×592 22.3 KB

As you can see, the folder is empty.

19

@peterjc123 Is there any requirement for the system to have pytorch installed in Python?

20

I tried going deeper into the error, it says it couldn’t find mnist.cpp file.

image
image981×180 7.55 KB