when saving leaf tensor using ctx.save_for_backward :
class DCNFunction(Function):
def forward(ctx, input, weight, Offset, Mask,
num_deformable_group,kernel, pad, stride, dilation,
use_bias , bias):
...
...
ctx.save_for_backward(input,weight, bias, Offset, Mask, col)
...
def backward(ctx, grad_out):
...
...
input, weight, bias, Offset, Mask, col = ctx.saved_tensors
...
Encountered an error：
File "dcnv2.py", line 54, in backward
input, weight, bias, Offset, Mask, col = ctx.saved_tensors
RuntimeError: No grad accumulator for a saved leaf!
weight and bias is self defined nn.Parameter , they are leaf node in graph
self.weight = torch.nn.Parameter(data=torch.empty(planes,inplanes,kernel,kernel),requires_grad= phase=='TRAIN')
ipdb> ctx.needs_input_grad
(True, True, True, True, False, False, False, False, False, False, True)
ipdb> ctx.next_functions
((<AccumulateGrad object at 0x7fa0fc5a0c18>, 0), (<AccumulateGrad object at 0x7fa0fc5a0c50>, 0), (<CudnnConvolutionBackward object at 0x7fa0fc5a0c88>, 0), (<MulBackward0 object at 0x7fa0fc5a0cc0>, 0), (<AccumulateGrad object at 0x7fa0fc5a0cf8>, 0))
when changing code(no error):
class DCNFunction(Function):
def forward(ctx, input, weight, Offset, Mask,
num_deformable_group,kernel, pad, stride, dilation,
use_bias , bias):
...
...
ctx.save_for_backward(input, Offset, Mask, col)
ctx.weight = weight
ctx.bias = bias
...
def backward(ctx, grad_out):
...
...
weight = ctx.weight
bias = ctx.bias
input, weight, bias, Offset, Mask, col = ctx.saved_tensors
...
using pytorch1.0
wonder why can’t save leaf tensor using ctx.save_for_backward？
and
comparing with ctx.save_for_backward, the disadvantage of using Temporary variables Eg.ctx.weight ?
thx