Add_event_handler arguments not passing correctly?

pytorchnewbie · July 21, 2020, 8:36am

Hi!

So when I do:

def log_engine_output(
    engine: Engine,
    fields: Dict[LOG_OP, Union[List[str], List[VisPlot], List[VisImg]]],
    epoch_num=None,
) -> None:
    """Log numerical fields in the engine output dictionary to stdout"""

    print(epoch_num)

    for mode in fields.keys():
        if (
            mode is LOG_OP.NUMBER_TO_VISDOM
            or mode is LOG_OP.VECTOR_TO_VISDOM
            or mode is LOG_OP.IMAGE_TO_VISDOM
        ):
            mode(
                engine,
                engine.state.vis,
                fields[mode],
                engine_attr="output",
                epoch_num=epoch_num,
            )
        else:
            mode(engine, fields[mode], engine_attr="output")


trainer.add_event_handler(
            Events.EPOCH_COMPLETED(every=1),
            log_engine_output,
            {
                # Log fields as message in logfile
                LOG_OP.LOG_MESSAGE: ["loss"],
                # Log fields as separate data files
                LOG_OP.SAVE_IN_DATA_FILE: ["loss"],
                # Plot fields to Visdom
                LOG_OP.NUMBER_TO_VISDOM: [
                    # First plot, key is "p1"
                    VisPlot(
                        "loss",
                        plot_key="p1",
                        split="nll_1",
                        title="Train loss: ",
                        x_label="Iters",
                        y_label="nll",
                        env=cfg.env,
                    ),
                ],
            },
            trainer.state.epoch,
        )

The epoch_num that is printed is always 0. That’s not right since it prints every epoch - why would this happen?

I’m also confusing about the structure of doing trainer.add_event_handler() vs. the decorator of
@trainer.on(Events.EPOCH_COMPLETED(every=1)).

I thought that the only difference between the decorator and the add_event_handler(some_function, variable1, variable2) was that with add_event_handler you could pass in the variables that are the arguments to the function (variable 1, variable 2), whereas with the decorator @train you can only pass in “engine” as your argument. Is this correct? If so, why isn’t passing the arguments working for me?

Also - what is the situation with add_event_handler automatically checking to see if the first argument it finds (after the function),is “engine”? From the tutorial, “The first argument can be optionally engine, but not necessary.”.

def log_metrics(engine, title):
    print("Epoch: {} - {} accuracy: {:.2f}"
           .format(trainer.state.epoch, title, engine.state.metrics["acc"]))

@trainer.on(Events.EPOCH_COMPLETED)
def evaluate(trainer):
    with evaluator.add_event_handler(Events.COMPLETED, log_metrics, "train"):
        evaluator.run(train_loader)

Here it automatically passes “evaluator” as the “engine” argument to log_metrics. However, here:

trainer = Engine(update_model)

trainer.add_event_handler(Events.STARTED, lambda _: print("Start training"))
# or
@trainer.on(Events.STARTED)
def on_training_started(engine):
    print("Another message of start training")
# or even simpler, use only what you need !
@trainer.on(Events.STARTED)
def on_training_started():
    print("Another message of start training")

# attach handler with args, kwargs
mydata = [1, 2, 3, 4]

def on_training_ended(data):
    print("Training is ended. mydata={}".format(data))

trainer.add_event_handler(Events.COMPLETED, on_training_ended, mydata)

The first argument is not “engine”. Does this mean that somehow ignite checks the function passed into add_event_handler to see if the first argument is called “engine”? And if so, does the nomenclature have to be exactly called “engine” in order for it to pass the engine that add_event_handler is attached to to the function?

Thanks!

vfdev-5 · July 21, 2020, 8:52am

Hi @pytorchnewbie

The epoch_num that is printed is always 0. That’s not right since it prints every epoch - why would this happen?

This a correct behaviour as passing trainer.state.epoch it is passed as value and not reference. I agree that it would be nice to have it as reference, but I have no idea how to do that in python.

You need to fetch epoch inside the handler with trainer.state.epoch.

vfdev-5 · July 21, 2020, 8:57am

I’m also confusing about the structure of doing trainer.add_event_handler() vs. the decorator of
@trainer.on(Events.EPOCH_COMPLETED(every=1)).

Normally and the intention is that both should be the same and provide the same functionnality. If it is not the case, it is a bug.

You can also pass variable using the decorator:

from ignite.engine import Engine, Events

trainer = Engine(lambda e, b: None)


@trainer.on(Events.ITERATION_COMPLETED, [0, 1, 2])
def print_my_data(_mydata):
    print("print_my_data : ", _mydata)
    
trainer.run([0])

vfdev-5 · July 21, 2020, 9:06am

Yes, internally, we check the signature of attached function and see if we can bind the function with and without the Engine.
There are following options possible:

with engine only

@trainer.on(event)
def handler(engine):
    assert engine == trainer

with engine and additional variables

@trainer.on(event, v1, v2)
def handler(engine, v1, v2):
    assert engine == trainer

without engine

@trainer.on(event)
def handler():
    pass

without engine, but with variables

@trainer.on(event, v1, v2)
def handler(v1, v2):
    pass

And if so, does the nomenclature have to be exactly called “engine” in order for it to pass the engine that add_event_handler is attached to to the function?

Yes, if you would like to pass engine, it has to be the first argument.
Hope it is more clear

pytorchnewbie · July 21, 2020, 9:34am

What does it mean passed by value and not reference? Value to me means at that moment what is the value of trainer.state.epoch, but isn’t this handler triggered completely anew every epoch? Therefore every epoch the value of trainer.state.epoch changes?

vfdev-5 · July 21, 2020, 9:36am

What does it mean passed by value and not reference?

It means that the function gets the value of trainer.state.epoch.

Therefore every epoch the value of trainer.state.epoch changes?

yes, it is updated internally by Engine during the run.

pytorchnewbie · July 21, 2020, 9:38am

To respond to : “Yes, internally, we check the signature of attached function and see if we can bind the function with and without the Engine.”

Strange… it doesn’t seem to work in this case:

def print_epoch(engine):
print("The training epoch is: " + str(engine.state.epoch))

trainer.add_event_handler(
Events.EPOCH_COMPLETED(every=1), print_epoch()
)

The error is : “TypeError: print_epoch() missing 1 required positional argument: ‘engine’”

I get the same error if I do:

trainer.add_event_handler(
Events.EPOCH_COMPLETED(every=1), print_epoch(), trainer
)

pytorchnewbie · July 21, 2020, 9:38am

Exactly… so then normally the value of trainer.state.epoch shouldn’t be 0 every single time the event handler is triggered?

vfdev-5 · July 21, 2020, 9:39am

It should be like that

def print_epoch(engine):
    print("The training epoch is: " + str(engine.state.epoch))

trainer.add_event_handler(
    Events.EPOCH_COMPLETED(every=1), print_epoch
)

print_epoch shouldn’t be called on the same line as trainer.add_event_handler.

pytorchnewbie · July 21, 2020, 9:40am

Ofcourse silly mistake I apologize

vfdev-5 · July 21, 2020, 9:44am

Well, it is more like the following:

def foo(x):
    print(x)
    

handlers = []

def add_handler(fn, args):
    handlers.append((fn, args))
    
    
def execute():
    for h, args in handlers:
        h(args)
        

epoch = 0

# args are registered with value 0
add_handler(foo, epoch)

execute()

# even if we update the variable, registered args are not changed
epoch = 1

execute()