Hi,
We do release the GIL as soon as we get out of python code. So for pytorch ops, it’s more or less all of them.
The backward, unless you implement custom Functions, will run completely out of the GIL.
2 Likes