Single weight-sharing across a network

I am not sure if you can share part of the tensor. Try How to use shared weights in different layers of a model and see if that works if you apply it to your case. I am curious too.