How does NoGradGuard works in Cpp?

C++
1

This is more a conceptual question… Consider:

NoGradGuard guard;
b -= lr * b.grad();

I know that NoGradGuard will disable action recording for the next gradient calculation.
But how does it really works?

2

Hi,

It actually sets a global flag that is checked by the backend whenever an op is done on a variable.
The guard itself saved the current status and set it to false in the constructor. And restore the saved status in it’s destructor. That way it is similar to a with torch.no_grad(): block in python.

4 Likes