UPDATE:
For anybody still encountering this issue, there is still no official solution quite yet as of 6/5/2024 except to just use CUDA 11.8 on Windows, or to install CUDA 11.8 (making sure that you specifically check the box to install Nsight NVTX) and copy the nvToolsExt64_1.lib from 11.8 to 21.x’s \lib\x64 directory.
There is already a GitHub issue for this, and the PyTorch developers already have a pull request addressing this bug and another pull request specifically addressing the NVTX changes. Currently they are working to merge their fix for this (looks like they are just reviewing changes before merging) which will stop Cmake from attempting to look for the library in CUDA 12.x+.
The reason this is happening is because Nvidia stopped including NVTX as a compiled library in CUDA 12.x+ with NVTX version 3, which is great if you’re a developer as you only have to include it’s header files now. Nvidia explains this here. Unfortunately, the LibTorch developers did not have time to account for this for cmake, which is still looking for the compiled library. That’s why you have to steal the .lib file from 11.8 in order to build on 12.x+.
I’ve been stuck on this for a few weeks, so hopefully this helps somebody out.