Batch calls to different networks with same input

1

The standard batching method is to provide multiple inputs to a single network. Does PyTorch have support for batching calls with the same input to multiple networks, provided those networks have the same architecture, just different weights?

2

If you want to feed the same input (or batch of inputs) to multiple networks, you can create a list of modules and optimizers. Just use these inside the training, like you would use a single network. Something like this should work.

models = list([mode1, model2,...., model10])
optimizers = list([optimizer1, optimizer2,...., optimizer10])

for model in models:
     model.train()
            
for batch in train_loader:
     inputs, targets = batch['input'].to(device), batch['target'].to(device)
     for c in range(classnum):
          optimizers[c].zero_grad()
                
                             :
                             :
                             :
           outputs = models[c](each_input)
           loss = criterion(outputs, targets)        
           loss.backward()
           optimizers[c].step()

           running_loss += loss.item()
3

This runs each model on the input sequentially. What I’m looking for is a way to run multiple models on the same input in parallel on the GPU (the same way that one model can run on multiple inputs in parallel instead of sequentially).

4

You can read the data just once (with the DataLoader). Then, create multiple models. Next, push each one to the desired device via to('cuda:id') and simply pass the data to each model.
Now, the training happens on different devices, so it should be executed in parallel.
If you want to train a model using multiple devices, set CUDA_VISIBLE_DEVICES.