Working with videos of arbitrary length

[Tom_Ginsberg]

I found a very similar post here that answers my questions

How to process variable length sequence of images with CNN

Sure, each data sequence is of format [image_0, action_1, image_1, action_2, image_2, action_3, image_3…], and the task is to predict action sequence [action_1, action_,…, action_k] given image sequence [image_0, image_1,…, image_k]. The image sequence of a variable length k+1 in a batch, so I pad each sequence with zero images until sequence length is max_seq_len. The batched input is thus of shape (B, max_seq_len, C, H, W). My network uses a CNN model to embed each image into a feature vector…