Creating A Batch of Sparse Tensors

1

Let say I want to create a 3-dimensional tensor, where the first dimension is the batch size,

a[batch_index][sparse_dim1][sparse_dim2].

Thus I need a hybrid sparse tensor representation. The first dimension is a dense dimension. The second and the third dimensions are sparse dimensions.

However, from the documentation, it seems that the dense dimension must be after the sparse dimension? Then how I represent the hybrid tensor?

2

Hi,

You can consider batch as a sparse dimension. it just happens to have an entry for each of them.
So considering this as a completely sparse tensor will work.

1 Like
3

Could you provide an example?

At the moment, I have index vector i, such as follows;

i = torch.tensor([[1,2,3],[0,1,2]])

The value is True on the location of i in matrix of size 10x10.

However, the problem is that in each batch the size of i is different.
For example, in one batch, it is 2x3. Another maybe 2x5.

Could you recommend a proper tools for this ?