GRU input as matrix

I am using GRU model for many to one prediction.
My input is represented at each time step as vector with length N, while using one feature, dim: (N, 1), The model is working as expected with dimension of (BATCH_SIZE, SEQ_LEN, N).
I want to add new feature for each point in the N rows, dimension: (N, 2), which will change the input dim to: (BATCH_SIZE, SEQ_LEN, N, 2) -How can I feed it to the GRU model? The model expected to (BATCH_SIZE, SEQ_LEN, N), however, if I use 2*N it won’t be good by the problem’s logical.


If you have some additional input structure, it is best to learn some fixed size 1d “summaries” before RNN, as none of standard implementations supports input groups or structures beyond vectors.