Non-ML models with ignite

ignite
1

In research, a lot of the time we will be comparing machine learning approaches to non-machine learning approaches. For the latter, there isn’t usually a notion of “iterations” or “epochs”. The model does what it does and outputs an error. Since the idea would be to code all the experiments in the same framework, I was wondering how to use ignite for non-learning based approaches/if it makes sense to do that. For instance, does it make sense to use ignite’s “metrics” class for a non-learning based approach? Does using any of ignite make sense for non-learning approaches?

Thanks!

2

Hi @pytorchnewbie

The model does what it does and outputs an error. Since the idea would be to code all the experiments in the same framework, I was wondering how to use ignite for non-learning based approaches/if it makes sense to do that. For instance, does it make sense to use ignite’s “metrics” class for a non-learning based approach? Does using any of ignite make sense for non-learning approaches?

Probably, it depends on details etc (if you could provide some, it would help), but I think this can be just considered as inference phase: a model (any kind of) is used to compute predictions over the batches of the test dataset and we wish to save the outputs

def inference_step(engine, batch):
    sample, meta_data, what_ever_data = batch
    result = non_ml_model(sample)
    return {
        "result": result,        
    }

inference_engine = Engine(inference_step)

@inference_engine.on(Events.ITERATION_COMPLETED)
def save_result(engine):
    sample, meta_data, what_ever_data = engine.state.batch
    result = engine.state.output["result"]
    # do something to save the result

Another example of using Engine can be to compute mean/std of the dataset (no ML-model at all):

from ignite.metrics import Average

def compute_mean_std(engine, batch):
    b, c, *_ = batch['image'].shape
    data = batch['image'].reshape(b, c, -1).to(dtype=torch.float64)
    mean = torch.mean(data, dim=-1).sum(dim=0)
    mean2 = torch.mean(data ** 2, dim=-1).sum(dim=0)
    return {"mean": mean, "mean^2": mean2}

compute_engine = Engine(compute_mean_std)
img_mean = Average(output_transform=lambda output: output['mean'])
img_mean.attach(compute_engine, 'mean')
img_mean2 = Average(output_transform=lambda output: output['mean^2'])
img_mean2.attach(compute_engine, 'mean2')
state = compute_engine.run(train_loader)
state.metrics['std'] = torch.sqrt(state.metrics['mean2'] - state.metrics['mean'] ** 2)
mean = state.metrics['mean'].tolist()
std = state.metrics['std'].tolist()

Let me know if this helps. Otherwise, please provide a minimal code snippet to understand better the question…

3

Thank you so much! I am figuring out something about the non-ml model and will respond to this in a day or two with more details.