General question. I ran an experiment to test the number of bytes used in sparse matrices versus nn.Linear layers of the same size. I set both nn.Linear layer and sparse matrix to all zeros and found that nn.Linear is twice as large as a sparse matrix. Why is that?

1 x 1,000,000,000 matrix

nn.Linear 8.0 Gb

sparse.FloatTensor 4.0 Gb

You can find my code here