Multi-gpu parallalization for online learning

Hello everyone,

I’m newbie to pytorch. i’m working currently on untrimmed video classification. the network i’m implementing does online learning which means a batch size of 1. In addition, the loss is calculated on all frames of a single video (the loss is equal to the mean of cross entropy across all frames).
the tutorial : DataParallel — PyTorch 1.8.1 documentation was not
of a great help since my batch size is 1 so the network runs only on GPU 1.

Hey @takieddine_soualhi I am trying to understand the use case.

In addition, the loss is calculated on all frames of a single video (the loss is equal to the mean of cross entropy across all frames).

Is it possible to treat each frame as a sample, so that the number of frames in the video will be the batch size? If that’s possible, then you can let each GPU to take care of a subset of the frames?