Inefficiency in metric calculation

ignite
1

Suppose I have two metrics, mse and correlation. Every 2 epochs of the training, I want to run the evaluator and collect the MSE metric. Also, at the end of training (800 epochs later), I want to run the evaluator and collect and the correlation metric. However, the metrics are attached to the evaluator engine. Therefore, every time evaluator is run, both metrics are calculated. This seems inefficient, since I only need to calculate correlation on the evaluator run at the end of the 800 training epochs, not 400 times (800/2) during training. Thoughts?

2

You can attach/detach metrics, so I think you can just attach it at the end of the training:

@trainer.on(Events.EPOCH_COMPLETED(every=2))
def compute_mse():
    evaluator.run(validation)

@trainer.on(Events.COMPLETED)
def compute_correlation():
    correlation_metric.attach(evaluator, "corr")
    evaluator.run(validation)
    # optionally
    correlation_metric.detach(evaluator)
1 Like
3

Beautiful! This is exactly what I wanted to know. Thanks!

1 Like