Hi,
I have a doubt regarding gradient descent. The below function tires to approximate the value of w and b. After backpropagating the value for w.grad(w is initialized with 1.0) should have been a negative number but I am receiving high positive number.
A negative grad value indicates that w should be increased while a positive number number indicates that w should be reduced.
Am I missing something? Please help me resolve this query. Thank you.
import torch
x = torch.rand([20, 1], requires_grad=True)
y = 3*x - 2
w = torch.tensor([1.], requires_grad = True)
b = torch.tensor([1.], requires_grad = True)
y_hat = w*x + b
loss = torch.sum((y_hat - y)**2)
print(loss) # tensor(79.0468, grad_fn=<SumBackward0>)
loss.backward()
print(w.grad, b.grad) # tensor([35.0812]) tensor([76.0854])