Here an example:
aa = torch.nn.functional.log_softmax(torch.autograd.Variable(torch.randn(4, 3), requires_grad=1), 1)
b = torch.diag(aa, 1)
c = torch.sum(b)
c.backward()
> RuntimeError Traceback (most recent call last) > <ipython-input-30-4857ca7423f9> in <module>() > ----> 1 c.backward() > > /usr/local/lib/python2.7/dist-packages/torch/autograd/variable.pyc in backward(self, gradient, retain_graph, create_graph, retain_variables) > 165 Variable. > 166 """ > --> 167 torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables) > 168 > 169 def register_hook(self, hook): > > /usr/local/lib/python2.7/dist-packages/torch/autograd/__init__.pyc in backward(variables, grad_variables, retain_graph, create_graph, retain_variables) > 97 > 98 Variable._execution_engine.run_backward( > ---> 99 variables, grad_variables, retain_graph) > 100 > 101 > > RuntimeError: output and gradOutput shapes do not match: output [4 x 3], gradOutput [3 x 3] at /data/packages/pytorch/torch/lib/THNN/generic/LogSoftMax.c:73
however, when the matrix a
is square I dont get the error.