Error building pytorch: Cannot open include file: 'pybind11/pybind11.h'

Hello experts, I am frustrated with this error when building Pytorch 1.4.0 with CUDA 10.2 on Windows 10.
The complete messages are:

FAILED: caffe2/CMakeFiles/caffe2_pybind11_state.dir/python/pybind_state_registry.cc.obj
C:\PROGRA~2\MICROS~2\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\cl.exe  /nologo /TP -DAT_PARALLEL_OPENMP=1 -DMAGMA_V2 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_DEPRECATE=1 -D_OPENMP_NOFORCE_MANIFEST -Dcaffe2_pybind11_state_EXPORTS -Iaten\src -I..\aten\src -I. -I..\ -I..\cmake\..\third_party\benchmark\include -I..\third_party\onnx -Ithird_party\onnx -I..\third_party\foxi -Ithird_party\foxi -Icaffe2\aten\src\TH -I..\aten\src\TH -Icaffe2\aten\src -I..\aten\..\third_party\catch\single_include -I..\aten\src\ATen\.. -Icaffe2\aten\src\ATen -I..\third_party\miniz-2.0.8 -I..\caffe2\core\nomnigraph\include -I"C:\Program Files\NVIDIA Corporation\NvToolsExt\include" -I..\caffe2\..\torch\csrc\api -I..\caffe2\..\torch\csrc\api\include -I..\c10\.. -I..\c10\cuda\..\.. -I..\cmake\..\third_party\googletest\googlemock\include -I..\cmake\..\third_party\googletest\googletest\include -I..\third_party\protobuf\src -ID:\workspace_torch\build_torch\mkl\include -I..\cmake\..\third_party\eigen -IC:\Miniconda3\envs\to140\include -IC:\Miniconda3\envs\to140\lib\site-packages\numpy\core\include -IC:\ci\pybind11_1627061579810\work\pybind11\include -I..\cmake\..\third_party\cub -ID:\workspace_torch\build_torch\magma\include -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\include" /DWIN32 /D_WINDOWS /GR  /w /EHa /MP /bigobj -openmp -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION /MD /O2 /Ob2  /w /EHa /MP /bigobj -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /Z7 /EHa /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -openmp /showIncludes /Focaffe2\CMakeFiles\caffe2_pybind11_state.dir\python\pybind_state_registry.cc.obj /Fdcaffe2\CMakeFiles\caffe2_pybind11_state.dir\ /FS -c ..\caffe2\python\pybind_state_registry.cc
D:\workspace_torch\build_torch\pytorch-1.4.0\caffe2/python/pybind_state_registry.h(3): fatal error C1083: Cannot open include file: 'pybind11/pybind11.h': No such file or directory
[340/1335] Building CXX object caffe2\CMakeFiles\caffe2_pybind11_state.dir\python\pybind_state_dlpack.cc.obj
FAILED: caffe2/CMakeFiles/caffe2_pybind11_state.dir/python/pybind_state_dlpack.cc.obj
C:\PROGRA~2\MICROS~2\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\cl.exe  /nologo /TP -DAT_PARALLEL_OPENMP=1 -DMAGMA_V2 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_DEPRECATE=1 -D_OPENMP_NOFORCE_MANIFEST -Dcaffe2_pybind11_state_EXPORTS -Iaten\src -I..\aten\src -I. -I..\ -I..\cmake\..\third_party\benchmark\include -I..\third_party\onnx -Ithird_party\onnx -I..\third_party\foxi -Ithird_party\foxi -Icaffe2\aten\src\TH -I..\aten\src\TH -Icaffe2\aten\src -I..\aten\..\third_party\catch\single_include -I..\aten\src\ATen\.. -Icaffe2\aten\src\ATen -I..\third_party\miniz-2.0.8 -I..\caffe2\core\nomnigraph\include -I"C:\Program Files\NVIDIA Corporation\NvToolsExt\include" -I..\caffe2\..\torch\csrc\api -I..\caffe2\..\torch\csrc\api\include -I..\c10\.. -I..\c10\cuda\..\.. -I..\cmake\..\third_party\googletest\googlemock\include -I..\cmake\..\third_party\googletest\googletest\include -I..\third_party\protobuf\src -ID:\workspace_torch\build_torch\mkl\include -I..\cmake\..\third_party\eigen -IC:\Miniconda3\envs\to140\include -IC:\Miniconda3\envs\to140\lib\site-packages\numpy\core\include -IC:\ci\pybind11_1627061579810\work\pybind11\include -I..\cmake\..\third_party\cub -ID:\workspace_torch\build_torch\magma\include -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\include" /DWIN32 /D_WINDOWS /GR  /w /EHa /MP /bigobj -openmp -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION /MD /O2 /Ob2  /w /EHa /MP /bigobj -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /Z7 /EHa /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -openmp /showIncludes /Focaffe2\CMakeFiles\caffe2_pybind11_state.dir\python\pybind_state_dlpack.cc.obj /Fdcaffe2\CMakeFiles\caffe2_pybind11_state.dir\ /FS -c ..\caffe2\python\pybind_state_dlpack.cc
d:\workspace_torch\build_torch\pytorch-1.4.0\caffe2\python\pybind_state_dlpack.h(9): fatal error C1083: Cannot open include file: 'pybind11/pybind11.h': No such file or directory
[341/1335] Building CXX object caffe2\CMakeFiles\caffe2_pybind11_state.dir\python\pybind_state_nomni.cc.obj
FAILED: caffe2/CMakeFiles/caffe2_pybind11_state.dir/python/pybind_state_nomni.cc.obj
C:\PROGRA~2\MICROS~2\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\cl.exe  /nologo /TP -DAT_PARALLEL_OPENMP=1 -DMAGMA_V2 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_DEPRECATE=1 -D_OPENMP_NOFORCE_MANIFEST -Dcaffe2_pybind11_state_EXPORTS -Iaten\src -I..\aten\src -I. -I..\ -I..\cmake\..\third_party\benchmark\include -I..\third_party\onnx -Ithird_party\onnx -I..\third_party\foxi -Ithird_party\foxi -Icaffe2\aten\src\TH -I..\aten\src\TH -Icaffe2\aten\src -I..\aten\..\third_party\catch\single_include -I..\aten\src\ATen\.. -Icaffe2\aten\src\ATen -I..\third_party\miniz-2.0.8 -I..\caffe2\core\nomnigraph\include -I"C:\Program Files\NVIDIA Corporation\NvToolsExt\include" -I..\caffe2\..\torch\csrc\api -I..\caffe2\..\torch\csrc\api\include -I..\c10\.. -I..\c10\cuda\..\.. -I..\cmake\..\third_party\googletest\googlemock\include -I..\cmake\..\third_party\googletest\googletest\include -I..\third_party\protobuf\src -ID:\workspace_torch\build_torch\mkl\include -I..\cmake\..\third_party\eigen -IC:\Miniconda3\envs\to140\include -IC:\Miniconda3\envs\to140\lib\site-packages\numpy\core\include -IC:\ci\pybind11_1627061579810\work\pybind11\include -I..\cmake\..\third_party\cub -ID:\workspace_torch\build_torch\magma\include -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\include" /DWIN32 /D_WINDOWS /GR  /w /EHa /MP /bigobj -openmp -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION /MD /O2 /Ob2  /w /EHa /MP /bigobj -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /Z7 /EHa /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -openmp /showIncludes /Focaffe2\CMakeFiles\caffe2_pybind11_state.dir\python\pybind_state_nomni.cc.obj /Fdcaffe2\CMakeFiles\caffe2_pybind11_state.dir\ /FS -c ..\caffe2\python\pybind_state_nomni.cc
D:\workspace_torch\build_torch\pytorch-1.4.0\caffe2/python/pybind_state_registry.h(3): fatal error C1083: Cannot open include file: 'pybind11/pybind11.h': No such file or directory
[342/1335] Building CXX object caffe2\CMakeFiles\caffe2_pybind11_state.dir\python\pybind_state.cc.obj
FAILED: caffe2/CMakeFiles/caffe2_pybind11_state.dir/python/pybind_state.cc.obj
C:\PROGRA~2\MICROS~2\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\cl.exe  /nologo /TP -DAT_PARALLEL_OPENMP=1 -DMAGMA_V2 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_DEPRECATE=1 -D_OPENMP_NOFORCE_MANIFEST -Dcaffe2_pybind11_state_EXPORTS -Iaten\src -I..\aten\src -I. -I..\ -I..\cmake\..\third_party\benchmark\include -I..\third_party\onnx -Ithird_party\onnx -I..\third_party\foxi -Ithird_party\foxi -Icaffe2\aten\src\TH -I..\aten\src\TH -Icaffe2\aten\src -I..\aten\..\third_party\catch\single_include -I..\aten\src\ATen\.. -Icaffe2\aten\src\ATen -I..\third_party\miniz-2.0.8 -I..\caffe2\core\nomnigraph\include -I"C:\Program Files\NVIDIA Corporation\NvToolsExt\include" -I..\caffe2\..\torch\csrc\api -I..\caffe2\..\torch\csrc\api\include -I..\c10\.. -I..\c10\cuda\..\.. -I..\cmake\..\third_party\googletest\googlemock\include -I..\cmake\..\third_party\googletest\googletest\include -I..\third_party\protobuf\src -ID:\workspace_torch\build_torch\mkl\include -I..\cmake\..\third_party\eigen -IC:\Miniconda3\envs\to140\include -IC:\Miniconda3\envs\to140\lib\site-packages\numpy\core\include -IC:\ci\pybind11_1627061579810\work\pybind11\include -I..\cmake\..\third_party\cub -ID:\workspace_torch\build_torch\magma\include -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\include" /DWIN32 /D_WINDOWS /GR  /w /EHa /MP /bigobj -openmp -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION /MD /O2 /Ob2  /w /EHa /MP /bigobj -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /Z7 /EHa /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -openmp /showIncludes /Focaffe2\CMakeFiles\caffe2_pybind11_state.dir\python\pybind_state.cc.obj /Fdcaffe2\CMakeFiles\caffe2_pybind11_state.dir\ /FS -c ..\caffe2\python\pybind_state.cc
D:\workspace_torch\build_torch\pytorch-1.4.0\caffe2/python/pybind_state_dlpack.h(9): fatal error C1083: Cannot open include file: 'pybind11/pybind11.h': No such file or directory
[343/1335] Building CXX object caffe2\CMakeFiles\caffe2_pybind11_state.dir\python\pybind_state_int8.cc.obj
FAILED: caffe2/CMakeFiles/caffe2_pybind11_state.dir/python/pybind_state_int8.cc.obj
C:\PROGRA~2\MICROS~2\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\cl.exe  /nologo /TP -DAT_PARALLEL_OPENMP=1 -DMAGMA_V2 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_DEPRECATE=1 -D_OPENMP_NOFORCE_MANIFEST -Dcaffe2_pybind11_state_EXPORTS -Iaten\src -I..\aten\src -I. -I..\ -I..\cmake\..\third_party\benchmark\include -I..\third_party\onnx -Ithird_party\onnx -I..\third_party\foxi -Ithird_party\foxi -Icaffe2\aten\src\TH -I..\aten\src\TH -Icaffe2\aten\src -I..\aten\..\third_party\catch\single_include -I..\aten\src\ATen\.. -Icaffe2\aten\src\ATen -I..\third_party\miniz-2.0.8 -I..\caffe2\core\nomnigraph\include -I"C:\Program Files\NVIDIA Corporation\NvToolsExt\include" -I..\caffe2\..\torch\csrc\api -I..\caffe2\..\torch\csrc\api\include -I..\c10\.. -I..\c10\cuda\..\.. -I..\cmake\..\third_party\googletest\googlemock\include -I..\cmake\..\third_party\googletest\googletest\include -I..\third_party\protobuf\src -ID:\workspace_torch\build_torch\mkl\include -I..\cmake\..\third_party\eigen -IC:\Miniconda3\envs\to140\include -IC:\Miniconda3\envs\to140\lib\site-packages\numpy\core\include -IC:\ci\pybind11_1627061579810\work\pybind11\include -I..\cmake\..\third_party\cub -ID:\workspace_torch\build_torch\magma\include -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\include" /DWIN32 /D_WINDOWS /GR  /w /EHa /MP /bigobj -openmp -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION /MD /O2 /Ob2  /w /EHa /MP /bigobj -DCUDA_HAS_FP16=1 -DUSE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD /Z7 /EHa /DNOMINMAX /wd4267 /wd4251 /wd4522 /wd4838 /wd4305 /wd4244 /wd4190 /wd4101 /wd4996 /wd4275 /bigobj -openmp /showIncludes /Focaffe2\CMakeFiles\caffe2_pybind11_state.dir\python\pybind_state_int8.cc.obj /Fdcaffe2\CMakeFiles\caffe2_pybind11_state.dir\ /FS -c ..\caffe2\python\pybind_state_int8.cc
D:\workspace_torch\build_torch\pytorch-1.4.0\caffe2/python/pybind_state_dlpack.h(9): fatal error C1083: Cannot open include file: 'pybind11/pybind11.h': No such file or directory
[352/1335] Building CXX object caffe2\CMakeFiles\pow_test.dir\__\aten\src\ATen\test\pow_test.cpp.obj
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "setup.py", line 755, in <module>
    build_deps()
  File "setup.py", line 316, in build_deps
    cmake=cmake)
  File "D:\workspace_torch\build_torch\pytorch-1.4.0\tools\build_pytorch_libs.py", line 62, in build_caffe2
    cmake.build(my_env)
  File "D:\workspace_torch\build_torch\pytorch-1.4.0\tools\setup_helpers\cmake.py", line 335, in build
    self.run(build_args, my_env)
  File "D:\workspace_torch\build_torch\pytorch-1.4.0\tools\setup_helpers\cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "C:\Miniconda3\envs\to140\lib\subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '16']' returned non-zero exit status 1.

And this is my pybind11 include dir:

Python 3.7.10 (default, Feb 26 2021, 13:06:18) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pybind11
>>> pybind11.get_include()
'C:\\Miniconda3\\envs\\to140\\lib\\site-packages\\pybind11\\include'

Please help me. Thanks.

EDIT:
Just notice that ā€œ-Iā€ has this value IC:\ci\pybind11_1627061579810\work\pybind11\include. How come this path used? And how to change it?