I’m not 100% sure what you problem is. Typically, in case of RNNs going from batch_size = 1 to batch_size > 1 requires to handle sequence of different lengths. However, you say
I’m creating sequences of length 1
which is a bit odd in itself as it’s not really a sequence :). In you code snippet it looks like seq_len=4. So this is all a bit confusing.
Just in case the problem might be sequences of different lengths: My approach is to create batches with sequences of the same length. This is particularly for autoencoders easy since input and target sequences are the same. We talked about this extensively in this post. If it helps, my code for that is available on Github.