How Preprocess video and work with that?

vision
1

Hi! I’m new of this forum.
I’m doing a video classification, I don’t know how to load video and how to preprocess video from my dataset.
My dataset has 99 videos divide in 6 classes, my big problem is how to divide video in training, validation and test.How to pre-process data video and how to upload my video in PyTorch?
Thanks a lot!!

2

NVIDIA has a video loader for PyTorch: nvvl.

The splitting depends on your use case. Would it be possible to split the videos based on the classes or do you want to split the videos frame-wise?

1 Like
3

Thanks for answering me immediately.
I would like to divide them into frames so that I can process them as images using a rnn. The problem is that I do not know how to load them directly in python or process them since the videos are integral and already divided into classes. But to train them I have to do a shuffle,I think, at the same time I should keep the video separate for supervised learning, for the test. How can I start?
Thanks again

4

So as I understand your use case you would like to classify each frame separately.
The video sequence is therefore not important, i.e. each frame has the target of the video target?

If so, you could just load the video, e.g. with OpenCV, and save each frame of the video into separate folders.
Using these images, you could then train a supervised model on the data.

I would split the videos in a stratified fashion, i.e. approx. using the same class distribution for training, validation and test data.

5

I apologize if I explained myself badly.
I meant that to classify the video I need every frame or at least to train a RNN, I use one frame every 5s for example, and I use the other 4 to train an RNN (like LSTM).
My goal is to train an RNN that allows me to classify 6 classes in my dataset composed of 99 videos.
My problem is that I don’t know how to upload videos since I have 99 videos tagged(like label), but all together and not divided into train, test and validation.
I don’t know how to load videos in python and how to divide them into train, validation and test.
I assumed that to classify a video I have to divide it into frames, am I wrong? How would you do it?
Initially I would like to train it using the CPU.
Could you recommend some approach for this project?
Thanks for the answer!!

6

hello Andrea, can you try perhaps https://github.com/etienne87/pytorch-streamloader and tell me if that fits your need?