Let say I want to create a 3-dimensional tensor, where the first dimension is the batch size,

a[batch_index][sparse_dim1][sparse_dim2].

Thus I need a hybrid sparse tensor representation. The first dimension is a dense dimension. The second and the third dimensions are sparse dimensions.

However, from the documentation, it seems that the dense dimension must be after the sparse dimension? Then how I represent the hybrid tensor?

You can consider batch as a sparse dimension. it just happens to have an entry for each of them.
So considering this as a completely sparse tensor will work.