Loop on part of the network, with new inputs

Hi, I’m trying to do something, that I don’t know if possible, I have 2 parts in my networks:
1.3d cnn
2.another part (not important for now).

The 3dcnn gets 4 frames, and outputs a vector of 256, which represents the 4 frames.
Now I want to use a big batch, because I want to train my second part of the network (the cnn is pretrained), but I don’t have enough memory.
The problem is the input to the cnn is too big, so I thought of doing a loop only on the cnn, and each loop instead of a batch of 64 I’ll do a batch of 16 (just an example).
but that means that I’m getting new inputs , while my second part is not running , which I don’t think is possible.
Another option I thought is to load the batch of 64 (means 64*4=256 frames), save it somewhere and then run few each time in a loop, but I’m not sure that I’ll have memory for this either.

Hope that you’ll have something in mind, Thanks a lot!