My impression about
autograd in pytorch is
When you only apply functions that support autograd, then the backward is automatically done with no extra efforts.
a = Variable(torch.randn(4,5,5,5), requires_grad=True)
# Is this ok?
a[:,:,1,1]=2 is equivalent to
All the functions that take
Variable as input do support autograd.
If you do this, keep in mind that
a[:,:,1,1].grad will always be equal to 0. Since it’s value is not used, it’s gradient is 0.
Hi, let me make questions more clear:
- As for
for-loop , does it supports autograd if it looks like this:
a = Variable(torch.randn(N,C,5,5), requires_grad=True)
conv = torch.nn.Conv2d(C, C, 3, stride=1, padding=1)
conv.weight.data = torch.randn(N, C, 3, 3)
for i in range(5):
a = conv(a) # if the result overide the original a
# if `a=conv(a)` is wrong, how can I do it?
When you do
a = conv(a), what happens is that you give the Tensor contained in the python variable
a to the convolution, get a new Tensor as output and use the python variable
a to refer to it.
This is not an in-place change of a, here you change what the python variable
a refers to.