Expand 2D sparse Tensor to do multiplication with 3D tensor

Hello,
I am trying to do a graphconv operation for a 3D data. Data ‘x’ is in shape [batch,seq_dim,feature_dim]. I have a sparse adjacency matrix ‘adj’ of shape [seq_dim,seq_dim]. I want to multiply adjacency matrix with input for all the samples. So i simply try to expand the adjacency matrix as adj.unsqueeze(0).expand([x.shape[0],-1,-1]). However, i get following error at this operation

RuntimeError: sparse tensors do not have strides

This error occurs at .expand operation. Unsqueeze() operation does not trigger this error. Please let me know if there is any other way around to do this operation with pytorch as well?

@ptrblck quick help will be much appreciated :slight_smile:

This is just a workaround, maybe someone has a better solution.

Here is a list with the available linear algebra operations with sparse matrices.

However some of them do not support batch multiplication yet. (github issue)

In this issue they mention that bmm can be used even with sparse matrices. However, it does not support broadcasting as shown in the docs. And sparse tensors cannot use the repeat method.

So the solution posted here is to stack your sparse tensors in the desired dimension.

This should work.

batch = 3
seq_dim = 150
feature_dim = 300

i = [[0, 1, seq_dim-1], [0, 2, seq_dim-1]]
v =  [3, 4, 5]
adj = torch.sparse_coo_tensor(i, v, dtype=torch.float)
new_adj = torch.stack([adj for _ in range(3)], dim=0)

x = torch.rand(batch, seq_dim, feature_dim)

print(torch.bmm(input=new_adj, mat2=x).shape)
1 Like