Sparse x Dense -> Dense matrix multiplication

1

Hi everyone,

I am trying to implement graph convolutional layer (as described in Semi-Supervised Classification with Graph Convolutional Networks) in PyTorch.

For this I need to perform multiplication of the dense feature matrix X by a sparse adjacency matrix A (sparse x dense -> dense). I don’t need to compute the gradients with respect to the sparse matrix A.

As mentioned in this thread, torch.mm should work in this case, however, I get the
TypeError: Type torch.sparse.FloatTensor doesn't implement stateless method addmm

class GraphConv(nn.Module):
    def __init__(self, size_in, size_out):
        super(GraphConv, self).__init__()
        self.W = nn.parameter.Parameter(torch.Tensor(size_in, size_out))
        self.b = nn.parameter.Parameter(torch.Tensor(size_out))

    def forward(self, X, A):
        return torch.mm(torch.mm(A, X), self.W) + self.b

A  # torch.sparse.FloatTensor, size [N x N]
X  # torch.FloatTensor, size [N x size_in]
A = torch.autograd.Variable(A, requires_grad=False) 
X = torch.autograd.Variable(X, requires_grad=False)
# If I omit the two lines above, I get invalid argument error (torch.FloatTensor, Parameter) for torch.mm

gcn = GraphConv(X.size()[1], size_hidden)
gcn(X, A)  # error here

Is there something I am doing wrong, or is this functionality simply not present in PyTorch yet?

1 Like
2

Hi Oleksandr - its a little hard to tell from your post which mm is actually triggering the error. Could you possibly provide a script for that triggers the issue? We do support mm for sparse x dense I believe.

3

Here is a simple script that reproduces the error: https://gist.github.com/shchur/5bf7eca2e68a44ae5c15947b553f9f1e

I get the following error when running this code

Screenshot from 2017-08-11 16-30-59
Screenshot from 2017-08-11 16-30-591337×581 82.3 KB

4

Here is an even clearer example

5

I think there are a few things here:

  1. I believe the issue with the first thing is that internally when calling mm on Variables, it calls torch.addmm and we don’t have the proper function defined on Sparse Tensors. (We actually have this implemented, but I think its named incorrectly)
  2. I’m less certain about nn.Parameter, I’ll have to let someone else answer that.

I will file an issue for #1.

1 Like
6

Thanks a lot, Trevor!

7

Any updates on this?

8

Here is the GitHub issue on this topic https://github.com/pytorch/pytorch/issues/2389. Apparently, this functionality is not supported at the moment, so we should wait for the feature to be added.

9

There is already a patch for torch.mm. You can use that in the meantime. I am using it currently and it if working fine.

10

Has there been any updates on doing sparse x dense operation with Variables?

I can do the following,

import torch
from torch.autograd import Variable as V

i = torch.LongTensor([[0, 1, 1], [1 ,1 ,1]])
v = torch.FloatTensor([3, 4, 5])
m = torch.sparse.FloatTensor(i, v, torch.Size([2,3]))
m2 = torch.randn(3,2)
print(torch.mm(m, m2))

But adding this breaks it,

M = V(m)
M2 = V(m2)
print(torch.mm(M, M2))

Traceback (most recent call last):
  File "test_sparse.py", line 11, in <module>
    print(torch.mm(M, M2))
RuntimeError: Expected object of type Variable[torch.sparse.FloatTensor] but found type Variable[torch.FloatTensor] for argument #1 'mat2'
1 Like
11

I converted it to a np.float32.

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import scipy.sparse as sp

n_samples = 100
n_features = 20
A_ = sp.random(n_samples, n_samples, dtype=np.float32)
X_ = np.random.random([n_samples, n_features]).astype(np.float32)
#↑here

def to_torch_sparse_tensor(M):
    M = M.tocoo().astype(np.float32)
    indices = torch.from_numpy(np.vstack((M.row, M.col))).long()
    values = torch.from_numpy(M.data)
    shape = torch.Size(M.shape)
    T = torch.sparse.FloatTensor(indices, values, shape)
    return T

X = torch.from_numpy(X_)
A = to_torch_sparse_tensor(A_)

class GraphConv(nn.Module):
    def __init__(self, size_in, size_out):
        super(GraphConv, self).__init__()
        self.W = nn.parameter.Parameter(torch.Tensor(size_in, size_out))
        self.b = nn.parameter.Parameter(torch.Tensor(size_out))
        # Initialize weights
        variance = 2 / (size_in + size_out)
        self.W.data.normal_(0.0, variance)
        self.b.data.normal_(0.0, variance)
    
    def forward(self, X, A):
        return torch.mm(torch.mm(A, X), self.W) + self.b

gcn = GraphConv(X.size()[1], 128)
gcn(X, A)

tensor([[ 2.5033e-02, 3.0702e-02, -4.3435e-02, …, 3.1698e-02,
-1.2819e-02, 4.3160e-02],
[ 1.3049e-02, -3.4871e-03, -2.6543e-02, …, 8.5519e-03,
7.0735e-03, 1.4211e-02],
[ 5.3494e-02, 3.7187e-02, -5.0533e-02, …, 2.2295e-02,
-1.8713e-02, 4.8472e-02],
…,
[ 2.3762e-02, 7.9290e-02, -6.0602e-02, …, 6.9588e-02,
-9.7072e-02, 1.3955e-01],
[ 3.3650e-02, 3.1555e-02, -2.4376e-02, …, 2.0983e-02,
2.0457e-03, 5.1157e-02],
[ 1.4304e-02, -1.9657e-02, -1.5910e-02, …, 3.1639e-03,
9.0677e-03, 8.1202e-03]])