I looked at the other threads with this problem but couldn’t find a solution.
My main file looks like this:
#include <torch/torch.h>
#include <iostream>
int main() {
torch::Device device(torch::kCUDA);
//torch::Tensor tensor = torch::eye(3); // (A)
torch::Tensor tensor = torch::eye(3, device); // (B)
std::cout << tensor << std::endl;
}
Running this gives me
Microsoft C++ exception: c10::NotImplementedError at memory location 0x000000DAA95CF0A0
Visual Studio is popping that error in the file variable_factories.h
(line 190) return autograd::make_variable(at::eye(n, at::TensorOptions(options).requires_grad(c10::nullopt)), /*requires_grad=*/options.requires_grad());
If I comment out line (B) and use line (A) instead, it works and prints
1 0 0
0 1 0
0 0 1
[ CPUFloatType{3,3} ]
Here are my include directories:
(ProjectDir)\libtorch-win-shared-with-deps-1.10.2+cu113\libtorch\include\torch\csrc\api\include
(ProjectDir)\libtorch-win-shared-with-deps-1.10.2+cu113\libtorch\include
(ProjectDir)\opencv\include
and the linked libraries:
torch.lib
torch_cpu.lib
torch_cuda.lib
torch_cuda_cu.lib
torch_cuda_cpp.lib
asmjit.lib
c10.lib
c10_cuda.lib
caffe2_detectron_ops_gpu.lib
caffe2_module_test_dynamic.lib
caffe2_nvrtc.lib
clog.lib
cpuinfo.lib
dnnl.lib
fbgemm.lib
libprotobuf-lite.lib
libprotobuf.lib
libprotoc.lib
mkldnn.lib
Any idea how to solve this?
I cannot reproduce the issue using libtorch==1.10.2_cu113
on Linux and get:
1 0 0
0 1 0
0 0 1
[ CUDAFloatType{3,3} ]
as the output.
Maybe you are hitting this issue on Windows.
Thanks for your reply!
I have downloaded libtorch-win-shared-with-deps-1.10.2+cu113.zip
from the pytorch webpage, and I’m using the headers and libraries from that zip, so they should be from the same version. Maybe something is in my %PATH% that’s overriding the dynamic libraries, but the only candidate would be CUDA 11.5 that is installed on my system, and I have already tried removing it from my path, but the problem remains.
What I found out further:
If I use libtorch-win-shared-with-deps-1.10.2+cu102.zip
(instead of …cuda113) and use tensor.cuda()
it works! But it’s literally the only configuration where it works. If I change .cuda()
to torch::zeros(3, device)
it doesn’t, also using .cuda()
with the ...cuda113.zip
package does not work…
I found the solution in this github post:
opened 07:08AM - 17 Mar 21 UTC
closed 09:05AM - 17 Mar 21 UTC
module: windows
## 🐛 Bug
Trying to load JIT torchscript model in Windows 10 using precompil… ed torch 1.8.0 with CUDA 11.1 results in
```
Could not run 'aten::empty_strided' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build).
```
This was previously solved by adding `"-INCLUDE:?warp_size@cuda@at@@YAHXZ"` to the linker options of MSVC as suggested [here](https://github.com/pytorch/pytorch/issues/33435#issuecomment-685241862)
when moving to 1.8.0 I added `torch_cuda_cpp.lib` and `torch_cuda_cu.lib` to the linked libraries
The same torchscript works well in 1.7.1
## Environment
- PyTorch Version (e.g., 1.0): 1.8.0
- OS (e.g., Linux): Windows 10 (MSVC 2017)
- How you installed PyTorch (`conda`, `pip`, source): Precompiled binaries
- Build command you used (if compiling from source):
- Python version: N/A (Libtorch)
- CUDA/cuDNN version: 11.1
- GPU models and configuration: Quatro P1000
- Any other relevant information:
cc @peterjc123 @maxluk @nbcsm @guyang3532 @gunandrose4u @mszhanyi @skyline75489
I already had the linker option /INCLUDE:"?warp_size@cuda@at@@YAHXZ"
added when I wrote the OP, but additionally adding /INCLUDE:"?searchsorted_cuda@native@at@@YA?AVTensor@2@AEBV32@0_N1@Z"
under Project properties → Linker → Command Line
solved the problem.
Now the main function in the OP results in
1 0 0
0 1 0
0 0 1
[ CUDAFloatType{3,3} ]```
1 Like
faucozo
(Orphee)
April 7, 2022, 8:32am
7
Sorry to refresh the issue, just adding possible solution for other users
I actually had the same error from a model learned on GPU and trying to load on CPU, had c10::NotImplementedError .
By specifying the device the error is gone
torch::jit::script::Module module;
torch::Device device(torch::kCPU);
module = torch::jit::load("model.pt", device);
Hope it helps !