How Pytorch compute gradient for division operator

I’m trying to compute the gradient of 1/x without using Pytorch’s autograd. I use the formula grad(1/x, x) = -1/x**2 but when I compare my result with the gradient given by Pytorch’s autograd, they’re different.

Here is my code

a = torch.tensor(np.random.randn(), dtype=dtype, requires_grad=True)
loss = 1/a
print(a.grad - (-1/(a**2)))

The output is

tensor(5.9605e-08, grad_fn=<ThAddBackward>)

Can anyone explain why ?

import torch
a = torch.tensor(-0.0061, dtype=torch.float32, requires_grad=True)
loss = 1.0/a

print(a.grad - (-1/(a**2)))
1 Like

Can you tell me where to see such implementations of backward() of Pytorch builtin operators ?

It is just my guess… wait for Pytorch guys

they are wrote in cpp codes, If you can build pytorch from source, it will be easy to find the implementations.