I’m trying to use torch.autograd.grad with create_graph=True to calculate the gradient with respect to one parameter (Let’s say A) then calculate the gradient of the previous gradient with respect to other parameters (Let’s say B).
The thing is, A is dense and B is sparse. The following is an example.
x = torch.sparse_coo_tensor([, ], [1.], (2,2), requires_grad=True) y = torch.tensor([[1., -1.], [1., 1.]], requires_grad=True) n = torch.norm(torch.sparse.mm(x, y)) g = torch.autograd.grad(n, y, create_graph=True)
Then I get an error: " The backward pass for this operation requires the ‘self’ tensor to be strided, but a sparse tensor was given instead. Please either use a strided tensor or set requires_grad=False for ‘self’ "
I understand that torch.sparse.mm will produce SparseAddmmBackward grad_fn so it’s different from normal cases. But I can’t figure out a solution based on this fact.
I wonder if there is any way to solve this problem (To calculate the gradient of g with respect to x in the above code). Any suggestions will be greatly appreciated.