I’m now reimplementing the “Holistically-nested Edge Detection”(arXiv) with pytorch, which has multiple “side-outputs”.
So how can I backward CNN with multiple outputs ? This thread suggested to use
torch.autograd.backward([x1, x2], [grad_x1, grad_x2]), but I run into error with this solution, following is my error message:
Traceback (most recent call last): File "main.py", line 68, in <module> train() File "main.py", line 63, in train torch.autograd.backward(outputs, grads) File "/usr/local/lib/python2.7/dist-packages/torch/autograd/__init__.py", line 98, in backward variables, grad_variables, retain_graph) RuntimeError: CUDNN_STATUS_MAPPING_ERROR
My code can be summarized as follow:
outputs = model(data) loss, grads = some_numpy_tricks(outputs) torch.autograd.backward(outputs, grads) print "now loss=%f"%loss
outputs is a list containing multiple outputs of the model, and
grads is the list containing gradients of loss w.r.t each model output.
So why did error occur and what’s the correct solution for my case?