Say, there are three networks, A, B and C.
During inference, I want to load a batch of images from a dataset and feed it to A with batch size of 300px, feed it to B with batch size of 700px. Then, we takes the output batch from these networks and feed it to network C after some post-processing.
So, far I have considered the following:
- Use a different dataloader for each network, dlA, dl B etc, and in the inference loop feed batches to the GPU analyzer.
- Use a single dataloader and use transforms function to reshape them along the way after the forward pass for network A, forward pass for network B etc.