Time series of embedded vector using LSTM and CNN

Hi, I’m new here and I’ve started using PyTorch a month ago.
I’m facing a problem using EMR to predict clinical outcome, which is a binary target. I’ve done with the entity extraction fro the free text and semi structured fields.
For RNN, does it make sense that I prepare a N500 input for each time point for each sample? I saw in most examples in NLP, input at each t is a 5001 vector if the embedded size is 500. Can I input a matrix there? or list of list(array) make more sense? What if I want to add a variable of interval between time points for?