CUDA 13.3 Update No Compatibility

Isaac1 · August 28, 2026, 2:31pm

I downloaded CUDA 13.3 as there new update but now I cant run PyTorch cause the latest version you support is CUDA 13.2. Could you please update this?

ptrblck · August 28, 2026, 4:34pm

The PyTorch pip wheels won’t use your locally installed CUDA toolkit.
If you need to explicitly use CUDA 13.3 you would need to build form source.
If you are fine with a nightly binary and CUDA 13.4 you can install it as it’s already available using the CUDA 13.4 Developer Preview.

Geremia · August 29, 2026, 3:04am

Are you saying Pytorch 2.13.0 is incompatible with CUDA 13.3 but compatible with CUDA 13.4?

I don’t know if this is related, but I’m getting the following issues compiling Pytorch 2.13.0 with CUDA 13.3 that I didn’t have compiling Pytorch 2.12.0 with CUDA 13.2:

[1456/3938] Building CUDA object caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim128_bf16_causal_sm80.cu.o
FAILED: [code=2] caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim128_bf16_causal_sm80.cu.o 
/usr/bin/ccache /opt/cuda-13.3/bin/nvcc -forward-unknown-to-host-compiler -DCUTE_SM90_EXTENDED_MMA_SHAPES_ENABLED -DCUTLASS_ENABLE_SM90_EXTENDED_MMA_SHAPES=1 -DCUTLASS_ENABLE_TENSOR_CORE_MMA=1 -DFLASHATTENTION_DISABLE_ALIBI -DFLASHATTENTION_DISABLE_SOFTCAP -DFLASH_NAMESPACE=pytorch_flash -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_POSIX_FALLOCATE=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DUNFUSE_FMA -DUSE_EXTERNAL_MZCRC -D_FILE_OFFSET_BITS=64 -I/tmp/SBo/pytorch-v2.13.0/build/aten/src -I/tmp/SBo/pytorch-v2.13.0/aten/src -I/tmp/SBo/pytorch-v2.13.0/build -I/tmp/SBo/pytorch-v2.13.0 -I/tmp/SBo/pytorch-v2.13.0/nlohmann -I/tmp/SBo/pytorch-v2.13.0/moodycamel -I/tmp/SBo/pytorch-v2.13.0/third_party/NNPACK/include -I/tmp/SBo/pytorch-v2.13.0/third_party/NNPACK/src -I/tmp/SBo/pytorch-v2.13.0/third_party/cpuinfo/include -I/tmp/SBo/pytorch-v2.13.0/third_party/pthreadpool/include -I/tmp/SBo/pytorch-v2.13.0/third_party/FXdiv/include -I/tmp/SBo/pytorch-v2.13.0/third_party/psimd/include -I/tmp/SBo/pytorch-v2.13.0/third_party/FP16/include -I/tmp/SBo/pytorch-v2.13.0/third_party/pthreadpool/src -isystem /tmp/SBo/pytorch-v2.13.0/build/third_party/gloo -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/gloo -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/tensorpipe/third_party/libuv/include -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/googletest/googlemock/include -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/googletest/googletest/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/protobuf/src -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/eigen -isystem /tmp/SBo/pytorch-v2.13.0/third_party/ideep/include -isystem /usr/include/oneapi/dnnl -isystem /tmp/SBo/pytorch-v2.13.0/INTERFACE -isystem /tmp/SBo/pytorch-v2.13.0/third_party/nlohmann/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/concurrentqueue -isystem /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc -isystem /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/cutlass/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc/flash_attn/src -DLIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch -gencode arch=compute_75,code=sm_75 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda  -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUB_WRAPPED_NAMESPACE=at_cuda_detail -DDISABLE_CUSPARSE_DEPRECATED -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -DC10_NODEPRECATED -O3 -DNDEBUG -std=c++20 -Xcompiler=-fPIC -DTORCH_USE_LIBUV -DCAFFE2_USE_GLOO -MD -MT caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim128_bf16_causal_sm80.cu.o -MF caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim128_bf16_causal_sm80.cu.o.d -x cu -c /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim128_bf16_causal_sm80.cu -o caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim128_bf16_causal_sm80.cu.o
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(71): error: template constraint not satisfied
    };
     ^
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/std/array(398): note #3135-D: substitution of arguments "<cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id>" for requires-clause failed
  requires (is_same_v<_Tp, _Args> && ...)
                                  ^
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/std/array(398): note #3042-D: atomic constraint not constant
  requires (is_same_v<_Tp, _Args> && ...)
                                  ^
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/std/array(398): note #2721-D: expression cannot be interpreted
  requires (is_same_v<_Tp, _Args> && ...)
                                  ^

/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(66): error: no instance of constructor "cuda::std::__4::array" matches the argument list
            argument types are: (cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id)
    return ::cuda::std::array{
                             ^

/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(66): error: cannot deduce class template arguments
    return ::cuda::std::array{
           ^

/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(67): error: too many initializer values
      arch_id::sm_60, arch_id::sm_61, arch_id::sm_62, arch_id::sm_70, arch_id::sm_75, arch_id::sm_80,
                      ^

4 errors detected in the compilation of "/tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim128_bf16_causal_sm80.cu".
[1457/3938] Building CXX object caffe2/CMakeFiles/torch_cpu.dir/__/aten/src/ATen/Version.cpp.o
[1458/3938] Building CXX object caffe2/CMakeFiles/torch_cpu.dir/__/aten/src/ATen/ThreadLocalState.cpp.o
[1459/3938] Building CXX object caffe2/CMakeFiles/torch_cpu.dir/__/aten/src/ATen/Utils.cpp.o
[1460/3938] Building CUDA object caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim192_fp16_causal_sm80.cu.o
FAILED: [code=2] caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim192_fp16_causal_sm80.cu.o 
/usr/bin/ccache /opt/cuda-13.3/bin/nvcc -forward-unknown-to-host-compiler -DCUTE_SM90_EXTENDED_MMA_SHAPES_ENABLED -DCUTLASS_ENABLE_SM90_EXTENDED_MMA_SHAPES=1 -DCUTLASS_ENABLE_TENSOR_CORE_MMA=1 -DFLASHATTENTION_DISABLE_ALIBI -DFLASHATTENTION_DISABLE_SOFTCAP -DFLASH_NAMESPACE=pytorch_flash -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_POSIX_FALLOCATE=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DUNFUSE_FMA -DUSE_EXTERNAL_MZCRC -D_FILE_OFFSET_BITS=64 -I/tmp/SBo/pytorch-v2.13.0/build/aten/src -I/tmp/SBo/pytorch-v2.13.0/aten/src -I/tmp/SBo/pytorch-v2.13.0/build -I/tmp/SBo/pytorch-v2.13.0 -I/tmp/SBo/pytorch-v2.13.0/nlohmann -I/tmp/SBo/pytorch-v2.13.0/moodycamel -I/tmp/SBo/pytorch-v2.13.0/third_party/NNPACK/include -I/tmp/SBo/pytorch-v2.13.0/third_party/NNPACK/src -I/tmp/SBo/pytorch-v2.13.0/third_party/cpuinfo/include -I/tmp/SBo/pytorch-v2.13.0/third_party/pthreadpool/include -I/tmp/SBo/pytorch-v2.13.0/third_party/FXdiv/include -I/tmp/SBo/pytorch-v2.13.0/third_party/psimd/include -I/tmp/SBo/pytorch-v2.13.0/third_party/FP16/include -I/tmp/SBo/pytorch-v2.13.0/third_party/pthreadpool/src -isystem /tmp/SBo/pytorch-v2.13.0/build/third_party/gloo -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/gloo -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/tensorpipe/third_party/libuv/include -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/googletest/googlemock/include -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/googletest/googletest/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/protobuf/src -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/eigen -isystem /tmp/SBo/pytorch-v2.13.0/third_party/ideep/include -isystem /usr/include/oneapi/dnnl -isystem /tmp/SBo/pytorch-v2.13.0/INTERFACE -isystem /tmp/SBo/pytorch-v2.13.0/third_party/nlohmann/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/concurrentqueue -isystem /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc -isystem /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/cutlass/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc/flash_attn/src -DLIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch -gencode arch=compute_75,code=sm_75 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda  -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUB_WRAPPED_NAMESPACE=at_cuda_detail -DDISABLE_CUSPARSE_DEPRECATED -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -DC10_NODEPRECATED -O3 -DNDEBUG -std=c++20 -Xcompiler=-fPIC -DTORCH_USE_LIBUV -DCAFFE2_USE_GLOO -MD -MT caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim192_fp16_causal_sm80.cu.o -MF caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim192_fp16_causal_sm80.cu.o.d -x cu -c /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim192_fp16_causal_sm80.cu -o caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim192_fp16_causal_sm80.cu.o
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(71): error: template constraint not satisfied
    };
     ^
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/std/array(398): note #3135-D: substitution of arguments "<cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id>" for requires-clause failed
  requires (is_same_v<_Tp, _Args> && ...)
                                  ^
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/std/array(398): note #3042-D: atomic constraint not constant
  requires (is_same_v<_Tp, _Args> && ...)
                                  ^
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/std/array(398): note #2721-D: expression cannot be interpreted
  requires (is_same_v<_Tp, _Args> && ...)
                                  ^

/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(66): error: no instance of constructor "cuda::std::__4::array" matches the argument list
            argument types are: (cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id)
    return ::cuda::std::array{
                             ^

/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(66): error: cannot deduce class template arguments
    return ::cuda::std::array{
           ^

/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(67): error: too many initializer values
      arch_id::sm_60, arch_id::sm_61, arch_id::sm_62, arch_id::sm_70, arch_id::sm_75, arch_id::sm_80,
                      ^

4 errors detected in the compilation of "/tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim192_fp16_causal_sm80.cu".
[1461/3938] Building CUDA object caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim256_bf16_causal_sm80.cu.o
FAILED: [code=2] caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim256_bf16_causal_sm80.cu.o 
/usr/bin/ccache /opt/cuda-13.3/bin/nvcc -forward-unknown-to-host-compiler -DCUTE_SM90_EXTENDED_MMA_SHAPES_ENABLED -DCUTLASS_ENABLE_SM90_EXTENDED_MMA_SHAPES=1 -DCUTLASS_ENABLE_TENSOR_CORE_MMA=1 -DFLASHATTENTION_DISABLE_ALIBI -DFLASHATTENTION_DISABLE_SOFTCAP -DFLASH_NAMESPACE=pytorch_flash -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_POSIX_FALLOCATE=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DUNFUSE_FMA -DUSE_EXTERNAL_MZCRC -D_FILE_OFFSET_BITS=64 -I/tmp/SBo/pytorch-v2.13.0/build/aten/src -I/tmp/SBo/pytorch-v2.13.0/aten/src -I/tmp/SBo/pytorch-v2.13.0/build -I/tmp/SBo/pytorch-v2.13.0 -I/tmp/SBo/pytorch-v2.13.0/nlohmann -I/tmp/SBo/pytorch-v2.13.0/moodycamel -I/tmp/SBo/pytorch-v2.13.0/third_party/NNPACK/include -I/tmp/SBo/pytorch-v2.13.0/third_party/NNPACK/src -I/tmp/SBo/pytorch-v2.13.0/third_party/cpuinfo/include -I/tmp/SBo/pytorch-v2.13.0/third_party/pthreadpool/include -I/tmp/SBo/pytorch-v2.13.0/third_party/FXdiv/include -I/tmp/SBo/pytorch-v2.13.0/third_party/psimd/include -I/tmp/SBo/pytorch-v2.13.0/third_party/FP16/include -I/tmp/SBo/pytorch-v2.13.0/third_party/pthreadpool/src -isystem /tmp/SBo/pytorch-v2.13.0/build/third_party/gloo -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/gloo -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/tensorpipe/third_party/libuv/include -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/googletest/googlemock/include -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/googletest/googletest/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/protobuf/src -isystem /tmp/SBo/pytorch-v2.13.0/cmake/../third_party/eigen -isystem /tmp/SBo/pytorch-v2.13.0/third_party/ideep/include -isystem /usr/include/oneapi/dnnl -isystem /tmp/SBo/pytorch-v2.13.0/INTERFACE -isystem /tmp/SBo/pytorch-v2.13.0/third_party/nlohmann/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/concurrentqueue -isystem /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc -isystem /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/cutlass/include -isystem /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc/flash_attn/src -DLIBCUDACXX_ENABLE_SIMPLIFIED_COMPLEX_OPERATIONS -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch -gencode arch=compute_75,code=sm_75 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda  -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUB_WRAPPED_NAMESPACE=at_cuda_detail -DDISABLE_CUSPARSE_DEPRECATED -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -DC10_NODEPRECATED -O3 -DNDEBUG -std=c++20 -Xcompiler=-fPIC -DTORCH_USE_LIBUV -DCAFFE2_USE_GLOO -MD -MT caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim256_bf16_causal_sm80.cu.o -MF caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim256_bf16_causal_sm80.cu.o.d -x cu -c /tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim256_bf16_causal_sm80.cu -o caffe2/aten/src/ATen/CMakeFiles/flash_attention.dir/__/__/__/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim256_bf16_causal_sm80.cu.o
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(71): error: template constraint not satisfied
    };
     ^
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/std/array(398): note #3135-D: substitution of arguments "<cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id>" for requires-clause failed
  requires (is_same_v<_Tp, _Args> && ...)
                                  ^
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/std/array(398): note #3042-D: atomic constraint not constant
  requires (is_same_v<_Tp, _Args> && ...)
                                  ^
/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/std/array(398): note #2721-D: expression cannot be interpreted
  requires (is_same_v<_Tp, _Args> && ...)
                                  ^

/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(66): error: no instance of constructor "cuda::std::__4::array" matches the argument list
            argument types are: (cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id, cuda::__4::arch_id)
    return ::cuda::std::array{
                             ^

/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(66): error: cannot deduce class template arguments
    return ::cuda::std::array{
           ^

/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h(67): error: too many initializer values
      arch_id::sm_60, arch_id::sm_61, arch_id::sm_62, arch_id::sm_70, arch_id::sm_75, arch_id::sm_80,
                      ^

4 errors detected in the compilation of "/tmp/SBo/pytorch-v2.13.0/third_party/flash-attention/csrc/flash_attn/src/flash_bwd_hdim256_bf16_causal_sm80.cu".

Geremia · August 31, 2026, 3:20am

Fixed it for me:

Patch:

--- a/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h
+++ b/opt/cuda-13.3/targets/x86_64-linux/include/cccl/cuda/__device/arch_id.h
@@ -66 +66 @@
-  return ::cuda::std::array{
+  return ::cuda::std::array<arch_id, 22>{

Your arch_id.h’s exact path might be different.