# How to input image sequences to a CNN+LSTM? **URL:** \ **Category:** Uncategorized\ **Created:** [July 14, 2020, 12:47pm UTC](https://discuss.pytorch.org/t/how-to-input-image-sequences-to-a-cnn-lstm/89149 "2020-07-14T12:47:56Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![dllacer](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/50afbb/32.png) [@dllacer](https://discuss.pytorch.org/u/dllacer)\ **Post date:** [July 14, 2020, 12:47pm UTC](https://discuss.pytorch.org/t/how-to-input-image-sequences-to-a-cnn-lstm/89149/1 "2020-07-14T12:47:56Z")
Hi everyone! I have a neural network that starts with some convolutional layers, then an LSTM layer and finally some deconvolutional layers. Kind of encoder-decoder architecture with LSTM in the middle. A sample in my dataset is a sequence of 4 images with shape `[4, 3, H, W]`. So, when I want to use batches, with `batch_size=8` for example, the resulting tensor would have shape `[8, 4, 3, H, W]`. The problem is that Conv2d layer does not accept 5-dimensional tensors, it expects 4-dimensional tensors. I’ve tried to reshape the tensor like this: `x.view(-1, 3, H, W)`, resulting this shape: `[32, 3, H, W]`. But this is not correct for me, as sequences are not related to each other, since they are picked randomly. So sequence 1 (4 contiguous images) in the batch is not the previous to sequence 2 (other 4 contiguous images) in this batch. How can I feed my network with this 5-dimensional tensor in order to keep the batch information and also the sequence information? ---
**Author:** ![AladdinPerzon](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aladdinperzon/32/13045_2.png) [@AladdinPerzon](https://discuss.pytorch.org/u/AladdinPerzon)\ **Post date:** [July 14, 2020, 1:41pm UTC](https://discuss.pytorch.org/t/how-to-input-image-sequences-to-a-cnn-lstm/89149/2 "2020-07-14T13:41:57Z")
The CNN will only look at each image separate from the others, so it won’t look at them in a sequence in any way unless you would concatenate them in some way, like along the channels or something like that. It’s a little unclear what you’re trying to accomplish but from my understanding you want the CNN to do some feature extraction from the images and then you can use the lstm on those features. So what I’m thinking you could do is what you said which is process each image separate by doing `x.view(batch_size*4, 3, H, W)` and then when each image has been processed by the CNN you will have shape `(batch_size*4, C, H_new, W_new)` and you can reshape this to the lstm by doing `x.view(batch_size, 4, -1)` if you’re using `batch_first=True` for the lstm. The lstm will then process them in a sequence. ---
**Author:** ![dllacer](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/50afbb/32.png) [@dllacer](https://discuss.pytorch.org/u/dllacer)\ **Post date:** [July 14, 2020, 1:51pm UTC](https://discuss.pytorch.org/t/how-to-input-image-sequences-to-a-cnn-lstm/89149/3 "2020-07-14T13:51:06Z")
That’s totally true, it was just a reordering problem. Thank you so much for clarifying it for me. ---
**Author:** ![Bilel](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/f0a364/32.png) [@Bilel](https://discuss.pytorch.org/u/Bilel)\ **Post date:** [August 26, 2022, 11:30am UTC](https://discuss.pytorch.org/t/how-to-input-image-sequences-to-a-cnn-lstm/89149/4 "2022-08-26T11:30:01Z")
Can you tell me how you resolve it please, I really need it, how can we generate many labels from one sequence. Thank you in advance ---
**Author:** ![tejasri19](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/53a042/32.png) [@tejasri19](https://discuss.pytorch.org/u/tejasri19)\ **Post date:** [August 30, 2022, 11:04am UTC](https://discuss.pytorch.org/t/how-to-input-image-sequences-to-a-cnn-lstm/89149/5 "2022-08-30T11:04:33Z")
Hey @dllacer , can you please share your code of the dataloader and model architecture for the model you used?