hi, I am a newer of Pytorch. In recently, I encountered a problem about the function “torch.autograd.backward”. I have read the doc about it, while I cannot know what it means for one parameter. The doc write “The graph is differentiated using the chain rule. If any of variables are non-scalar (i.e…

In [12]: x = Variable(torch.randn(10), requires_grad=True) In [13]: y = x ** 2 In [14]: grad = torch.randn(10) In [15]: torch.autograd.backward([y], [grad])

Dear smth, Thank you very much! From what I understand, I think torch.autograd.backward([y], [grad]) is that we apply function grad to y. I do not know if it is right? To understand the function deeply, I construct two examples to show what I want to do. Firstly, I want to compute the derivation …

From what I understand, I think torch.autograd.backward([y], [grad]) is that we apply function grad to y. I do not know if it is right? No. This will compute the gradient of all leaf nodes in the graph that y was created by. Leaf nodes are user-created Variables. In this case the only leaf node i…

Dear smth, thank you very much for your help! And I understood your idea! I also find another related problem and wish it is helpful for others. Thanks! [image] Clarification - Using backward() on non-scalars Hey, Sorry if this is obvious, but I find the description of…

[image] smth: Since `y` is not a scalar here, we are giving the gradient wrt each output component y[0], y[1], y[2] etc. in the `grad` part of `torch.autograd.backward`. I hope that made it clearer. How do you initialize the [grad] vector? can we randomly initialize it?

You might initialize it randomly or with any other gradient value you would like to pass. However, the default value would be torch.ones.

How to use torch.autograd.backward when variables are non-scalar

autograd

matliu (Matliu) July 3, 2017, 12:05am 5

Dear smth, thank you very much for your help! And I understood your idea! I also find another related problem and wish it is helpful for others. Thanks!

How PyTorch differentiates on non-scalar variable?