How to do LSTM/RNN with many-to-many settings(delayed version)

Hi everyone,
I am using nn.LSTMCell, trying to do a many-to-many mapping (delayed version), where the first output is 3 time step later than the input.
My question is : The input is only available in time step 1 and there is no more input to LSTMCell from last 3 time step. So how shall I define the input for the last three time step (t_end-3) during training? And in future prediction, is there any special thing that I need to take care of?
Thanks