To my knowledge there isn’t an official way from libtorch to use CUDA Graphs like on the latest PyTorch release. In this case, is it still possible (& safe?) to use “vanilla” CUDA Graphs from the CUDA API as described here: Getting Started with CUDA Graphs | NVIDIA Developer Blog ? What would be safe to record using such an approach? Are .backward() and stepping through an optimizer safe to record? What about NoGradGuard or things like model->train() and model->eval()? Thanks !
I don’t know how exactly you want to use CUDA graphs directly, but would claim it’s prone to error due to the strict memory requirements. The better way might be to try out to use the backend API directly as defined here.
1 Like
Wow thank you, I have missed that there is a CUDA Graphs backend in the C++ api! (no entry in the C++ documentation though)