I previously had a use-case wherein I was training an auto-encoder that learned rank 4 tensors that modeled the weights between a large graph of words. The majority of the words shared no weights and were thus 0. I also needed to normalize the columns of each matrix (at the rank 2 level) of these tensors.
I found that very few of the basic tensor operations for dense vectors were implemented for sparse vectors (mm products, etc), and there was no easy way to normalize. I ended up needing to do a ton of hacky things to reformat my problem with dense vectors that were rank 3 in order to be able to feasibly run all of the computations.
Idk if this is too vague to be helpful…