I installed pytorch but than ran into a “OSError: CUDA_HOME environment variable is not set” error when I tried installing flash attention after that. I’m curious if the cuda toolkit is installed automatically when following pytorch installation steps in Start Locally | PyTorch. If it is installed, where it is installed?
If it’s not installed, some feedback is it would be helpful to mention that you need to install it separately.
Thanks for making pytorch!
The PyTorch binaries ship with their own CUDA runtime dependencies, but do not install a full CUDA toolkit (for development).
Your flash-attn build seems to fail as it’s expecting a full CUDA compiler to build this package from source, as the package ships with a source distribution only (not wheels).
Great answer, thanks! Unfortunately compiling flash-attn is taking forever (more than 3 hours!) so I’m going to have to switch tactics. I’m thinking I’ll switch from windows to WSL.