Engine run with multi inputs

ignite
1

I am solving multilabel segmentation task. I want to train model with images and tabular data. So i have custom model class with multi inputs

class ContextNet(Module):
  def __init__(self, classes=10, features=10, dropout=0.1):
    super(ContextNet, self).__init__()
....
  def forward(self, image, predictions):
....
    return

so i need data generator which returns image, tabular data and mask. What is correct way to do that?
When i do such generator:

class gen_context(Dataset):
    def __init__(self, root, val=False):
        self.val = val
        self.map = array(listdir(root + 'Images/'))

    def __len__(self):
        return len(self.map)

    def __getitem__(self, index):
...
        return image, tabular, mask

i got

  File "/EFSDATA/fermata_classification/src/modules/train/train.py", line 143, in main_trainer
    trainer.run(train_loader, max_epochs=Params.epochs_to_stop)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 704, in run
    return self._internal_run()
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 783, in _internal_run
    self._handle_exception(e)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 466, in _handle_exception
    raise e
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 753, in _internal_run
    time_taken = self._run_once_on_dataset()
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 854, in _run_once_on_dataset
    self._handle_exception(e)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 466, in _handle_exception
    raise e
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 840, in _run_once_on_dataset
    self.state.output = self._process_function(self, self.state.batch)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/__init__.py", line 101, in update
    x, y = prepare_batch(batch, device=device, non_blocking=non_blocking)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/__init__.py", line 37, in _prepare_batch
    x, y = batch
ValueError: too many values to unpack (expected 2)

in other variants i got

  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 840, in _run_once_on_dataset
    self.state.output = self._process_function(self, self.state.batch)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/__init__.py", line 102, in update
    y_pred = model(x)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: 'predictions'
2

Hi @Artur_Punch,

according to traceback line:

File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/__init__.py", line 102, in update

you should be using create_supervised_trainer which is a helper function to create a trainer for a simple supervised task.

In your case you can create a trainer directly using Engine API:

model = ...
model = model.cuda()
optimized = ...
criterion = ...


def train_step(engine, batch):
    model.train()
    optimizer.zero_grad()

    image, tabular, mask = batch[0].cuda(), batch[1].cuda(), batch[2].cuda()

    outputs = model(image, tabular)   # or whatever logic you have for your model
    loss = criterion(outputs, mask)
    loss.backward()
    optimizer.step()

    return loss.item()
    
trainer = Engine(train_step)

Let me know if it works for you?