Autograd.grad with create_graph=True for Sparse Tensor

autograd
1

I’m trying to use torch.autograd.grad with create_graph=True to calculate the gradient with respect to one parameter (Let’s say A) then calculate the gradient of the previous gradient with respect to other parameters (Let’s say B).

The thing is, A is dense and B is sparse. The following is an example.

x = torch.sparse_coo_tensor([[0], [1]], [1.], (2,2), requires_grad=True)
y = torch.tensor([[1., -1.], [1., 1.]], requires_grad=True)
n = torch.norm(torch.sparse.mm(x, y))
g = torch.autograd.grad(n, y, create_graph=True)

Then I get an error: " The backward pass for this operation requires the ‘self’ tensor to be strided, but a sparse tensor was given instead. Please either use a strided tensor or set requires_grad=False for ‘self’ "

I understand that torch.sparse.mm will produce SparseAddmmBackward grad_fn so it’s different from normal cases. But I can’t figure out a solution based on this fact.

I wonder if there is any way to solve this problem (To calculate the gradient of g with respect to x in the above code). Any suggestions will be greatly appreciated. :grinning:

2

Hi Ago!

Perhaps this stackoverflow thread will prove useful.

Best.

K. Frank

3

Hi Frank!

That thread is posted by me, but thanks for your reply.

4

I find this function can solve my problem.

x = torch.sparse_coo_tensor([[0], [1]], [1.], (2,2), requires_grad=True)
y = torch.tensor([[1., -1.], [1., 1.]], requires_grad=True)
x1 = x.coalesce()
n = torch.norm(torch_sparse.spmm(x1.indices(), x1.values(),
 x1.size()[0], x1.size()[1], y))
g = torch.norm(torch.autograd.grad(n, y, create_graph=True)[0])
g.backward()