libtorch 2.0.1 with CUDA 11.8 and cudNN 8.5 works perfectly fine for me in Visual Studio Community 2022.
However, i now wanted to upgrade to libtorch Preview (nightly) with CUDA 12.1 and cudNN 8.9, but now i get an error message when trying to build with cmake:
[CMake] …
[CMake] – Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/include (found version “12.1.105”)
[CMake] CMake Error at C:/…/libtorch/share/cmake/Caffe2/public/cuda.cmake:74 (message):
[CMake] Failed to find nvToolsExt.
I saw in NVTX github that the most recent NVTX version (nvtx3) is using a header only file instead of a library.
Could that be the reason for the error i get?
Is this a known bug or am i doing something wrong?
The error message you are encountering indicates that CMake is unable to find the nvToolsExt library, which is typically provided by the NVIDIA CUDA Toolkit. This library is used for GPU profiling and debugging.
The issue could be related to the version mismatch between your CUDA installation and the libtorch Preview version you are trying to use. It’s possible that the libtorch Preview version you have is expecting a different version of nvToolsExt than the one provided by CUDA 12.1.
To address this issue, you can try the following steps:
Confirm that the nvToolsExt library is indeed present in your CUDA 12.1 installation. Check the C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/lib64 directory for the library file (nvToolsExt64_1.lib on Windows).
Ensure that the CUDA 12.1 installation directory is correctly set in your system’s environment variables (PATH variable) or in the CMake configuration. This ensures that CMake can locate the necessary CUDA files.
If the library is present and the environment variables are correctly set, you can try manually adding the path to the nvToolsExt library in your CMakeLists.txt file. Add the following line before the find_package(Torch REQUIRED) line:
Replace the path with the correct location of the lib64 directory on your system.
If the above steps do not resolve the issue, it’s possible that there is an incompatibility between the libtorch Preview version and CUDA 12.1. In that case, you might need to wait for a newer libtorch Preview version that is compatible with CUDA 12.1 or consider downgrading your CUDA installation to a version that is compatible with the libtorch Preview version you want to use.
I hope this helps! Let me know if you have any further questions.
Thank you for the extensive answer. I am running into this problem too, but I cannot find a nvToolsExt lib anywhere under C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1. I ran the NVIDIA CUDA installer as specified on the download page and installed all possible packages.
How can I best install the nvToolsExt lib? It does not seem to come with the default installer
I had a similar problem and what I did was install NVTX with CUDA 11.8 installer. With CUDA 12 installed, run CUDA 11.8 installer and select custom installation. Then select Nsight NVTX alone and leave everything else unchecked.
I’ve just sign into this forum to say thank you. I was about to create an issue on pytorch’s github saying that link libraries paths to cuda didn’t match the installation of Nvidia’s new 12.3 toolkit, but your direction to install NVTX from an older version of the toolkit saved the day. I think Nvidia should include NVTX as well on newer versions.
Its not like this since the last time I looked at this since cuda 12 NVIDA dropped support of NVTX in favor of NVTX3 which can be found under …/CUDA/%version%/NvtoolsExt which isn’t NVTX but NVTX3. Pytorch tried to update/upgrade to NVTX3 from what I see in the github logs but it failed to go through all test and so they reverted the changes and nobody fixed it. What you have are maybe remnant of an old or other installation.
A bit a rant Pytroch is at the moment far from stable only the bugs which are with the big frameworks and libraries are getting fixed and there is so much stuff to get fixed it starts to be a drag and there is more coming python will come with real mulithreading a lot of stuff will break because Pytroch isn’t build around it and has many static functions e.g like seed() . It maybe take a bit longer but MS want to drop the console for powershell = old help scripts won’t work anymore also in not so far future the next VS version is coming.
For anybody still encountering this issue, there is still no official solution quite yet as of 6/5/2024 except to just use CUDA 11.8 on Windows, or to install CUDA 11.8 (making sure that you specifically check the box to install Nsight NVTX) and copy the nvToolsExt64_1.lib from 11.8 to 21.x’s \lib\x64 directory.
The reason this is happening is because Nvidia stopped including NVTX as a compiled library in CUDA 12.x+ with NVTX version 3, which is great if you’re a developer as you only have to include it’s header files now. Nvidia explains this here. Unfortunately, the LibTorch developers did not have time to account for this for cmake, which is still looking for the compiled library. That’s why you have to steal the .lib file from 11.8 in order to build on 12.x+.
I’ve been stuck on this for a few weeks, so hopefully this helps somebody out.
The issue is ongoing. Long story short: Nvidia did a number and changed the here mentioned library from binary (a lib file) to a header-only. libtorch still hasn’t caught up with this change leading to perpetual complains of the few who dare use libtorch (I myself am trying ONNX right now to get away from libtorch) about this issue.
The solution given by @AidanShipperley is no longer working since the binaries are no longer shipped. I checked with 11.8 and 12.2. The header can be found under <CUDA_PATH>\include\nvtx3.
However, I still have an older libtorch2.1.0 where the DLL is actually shipped and can be found (once you extract the libtorch ZIP archive) under lib subdirectory.
Sadly still an issue with PyTorch 2.5.0 with CUDA 12.2 (12.4 max) on Windows 11.
I tried fiddling with torch/share/cmake/Caffe2/public/cuda.cmake since I am using Visual Studio 2022 with CMake 3.26; nvToolsExt has been deprecated since 3.25. I did notice that the find_path call specifies NO_DEFAULT_PATH to suppress looking through system paths. However, the path "${PROJECT_SOURCE_DIR}/third_party/NVTX/c/include" isn’t one I can find in the PyTorch distribution I installed via pip. In fact, PROJECT_SOURCE_DIR doesn’t resolve to any directory inside the site-packages/torch but rather still refers to my project’s top-level directory. That’s a bit strange but I guess the Caffe2 find module doesn’t call project() directly.
Edit: I had a few questions I wanted to ask but commit 3cfd244 gives me some answers. I.e.:
Is there any reason NO_DEFAULT_PATH is being used?
This supports building LibTorch from source using the CMake build config to use the vendored third_party/NVTX submodule tracking the NVTX repo.
Can we relax this requirement on Windows?
Probably yes in general but only with an explicit CMake build flag option; this option would also be OFF to preserve existing source build behavior. 3cfd244 adds a USE_SYSTEM_NVTX CMake option to the LibTorch CMake config that could ostensibly be used in cuda.cmake if the user specified -DUSE_SYSTEM_NVTX=ON in their project.
And does /path/to/third_party/NVTX/c/include exist?
Yes, in the PyTorch source repo. This path should be hardcoded in cuda.cmake in the first place but unfortunately it’s there in the 2.5.0 release.
I suppose building LibTorch from source may be friendlier for those using CUDA 12+ but this is of course not yet in an official PyTorch pip release.
Good news is that I did find a user workaround that does not require tweaking any of the CMake modules that come with a PyTorch distribution. Assuming that PYTORCH_VERSION and PYTORCH_CUDA_VERSION are available, with these two CMake variables defined as values from running the following Python script and reading the output:
from torch import version
if __name__ == "__main__":
print(version.__version__.split("+")[0]) # for PYTORCH_VERSION
print(version.cuda) # for PYTORCH_CUDA_VERSION
Right before your find_package(Torch CONFIG) call you can do the following:
# assumes find_package(CUDAToolkit) was already done
if(
PYTORCH_VERSION VERSION_GREATER_EQUAL 2.5.0 AND
PYTORCH_CUDA_VERSION VERSION_GREATER_EQUAL 12
)
message(STATUS "PyTorch NVTX headers workaround: Yes")
# only do this if nvToolsExt is not defined and CUDA::nvtx3 exists
if(NOT TARGET CUDA::nvToolsExt AND TARGET CUDA::nvtx3)
add_library(CUDA::nvToolsExt INTERFACE IMPORTED)
# ensure that PyTorch is told to use NVTX3 headers
target_compile_definitions(
CUDA::nvToolsExt INTERFACE
TORCH_CUDA_USE_NVTX3
)
target_link_libraries(CUDA::nvToolsExt INTERFACE CUDA::nvtx3)
endif()
else()
message(STATUS "PyTorch NVTX headers workaround: No")
endif()
# find Torch C++ as usual
find_package(Torch CONFIG)
This has pacified CMake nicely for me so hopefully it will be of help to others too.
