c10::NotImplementedError with minimal example

I looked at the other threads with this problem but couldn’t find a solution.

My main file looks like this:

#include <torch/torch.h>
#include <iostream>

int main() {
	torch::Device device(torch::kCUDA);
	//torch::Tensor tensor = torch::eye(3); // (A)
	torch::Tensor tensor = torch::eye(3, device); // (B)
	std::cout << tensor << std::endl;
}

Running this gives me

Microsoft C++ exception: c10::NotImplementedError at memory location 0x000000DAA95CF0A0

Visual Studio is popping that error in the file variable_factories.h

(line 190) return autograd::make_variable(at::eye(n, at::TensorOptions(options).requires_grad(c10::nullopt)), /*requires_grad=*/options.requires_grad());

If I comment out line (B) and use line (A) instead, it works and prints

1  0  0
0  1  0
0  0  1
[ CPUFloatType{3,3} ] 

Here are my include directories:

(ProjectDir)\libtorch-win-shared-with-deps-1.10.2+cu113\libtorch\include\torch\csrc\api\include
(ProjectDir)\libtorch-win-shared-with-deps-1.10.2+cu113\libtorch\include
(ProjectDir)\opencv\include

and the linked libraries:

torch.lib
torch_cpu.lib
torch_cuda.lib
torch_cuda_cu.lib
torch_cuda_cpp.lib
asmjit.lib
c10.lib
c10_cuda.lib
caffe2_detectron_ops_gpu.lib
caffe2_module_test_dynamic.lib
caffe2_nvrtc.lib
clog.lib
cpuinfo.lib
dnnl.lib
fbgemm.lib
libprotobuf-lite.lib
libprotobuf.lib
libprotoc.lib
mkldnn.lib

Any idea how to solve this?

I cannot reproduce the issue using libtorch==1.10.2_cu113 on Linux and get:

 1  0  0
 0  1  0
 0  0  1
[ CUDAFloatType{3,3} ]

as the output.
Maybe you are hitting this issue on Windows.

Thanks for your reply!

I have downloaded libtorch-win-shared-with-deps-1.10.2+cu113.zip from the pytorch webpage, and I’m using the headers and libraries from that zip, so they should be from the same version. Maybe something is in my %PATH% that’s overriding the dynamic libraries, but the only candidate would be CUDA 11.5 that is installed on my system, and I have already tried removing it from my path, but the problem remains.

What I found out further:
If I use libtorch-win-shared-with-deps-1.10.2+cu102.zip (instead of …cuda113) and use tensor.cuda() it works! But it’s literally the only configuration where it works. If I change .cuda() to torch::zeros(3, device) it doesn’t, also using .cuda() with the ...cuda113.zip package does not work…

I found the solution in this github post:

I already had the linker option /INCLUDE:"?warp_size@cuda@at@@YAHXZ" added when I wrote the OP, but additionally adding /INCLUDE:"?searchsorted_cuda@native@at@@YA?AVTensor@2@AEBV32@0_N1@Z" under Project properties → Linker → Command Line solved the problem.

Now the main function in the OP results in

1  0  0
0  1  0
0  0  1
[ CUDAFloatType{3,3} ]```
1 Like

Sorry to refresh the issue, just adding possible solution for other users

I actually had the same error from a model learned on GPU and trying to load on CPU, had c10::NotImplementedError .

By specifying the device the error is gone

torch::jit::script::Module module;
torch::Device device(torch::kCPU);

module = torch::jit::load("model.pt", device);

Hope it helps !