I want to index a sparse tensor and propagate gradients through it.
Now, I implement it like this:
import torch
i = torch.tensor([[0, 1]])
v = torch.tensor([2., 2.], requires_grad=True)
s = torch.sparse_coo_tensor(i, v, size=[4], requires_grad=True)
print(s)
r = torch.index_select(s, 0, torch.tensor([1, 2])).to_dense()
print(r)
r.sum().backward()
print(v.grad)
But it returns the following error when call the “backward()”:
tensor(indices=tensor([[0, 1]]),
values=tensor([2., 2.]),
size=(4,), nnz=2, layout=torch.sparse_coo,
grad_fn=<SparseCooTensorWithDimsAndTensorsBackward0>)
tensor([2., 0.], grad_fn=<ToDenseBackward0>)
Traceback (most recent call last):
File "/home/pb/slidr/test.py", line 11, in <module>
r.sum().backward()
File "/home/pb/miniconda3/envs/pytorch/lib/python3.9/site-packages/torch/_tensor.py", line 396, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/pb/miniconda3/envs/pytorch/lib/python3.9/site-packages/torch/autograd/__init__.py", line 173, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: Tensors of type SparseTensorImpl do not have strides
I need a dense results after “index_select”, so I called the “to_dense()”. Is there a way to realize indexing through a list for sparse tensors and support back-propagation?