When to use Module based loss functions vs functional loss functions

I’m usually creating the criterion as a module in case I want to store some internal states, e.g. weight, a specific reduction etc.
Also, I would say it basically depends on your coding style and the use case you are working with.
E.g. if you are reusing the criterion in multiple places (e.g. GAN training) and would like to experiment with different loss functions as well as reductions, a single instance might be more convenient to use instead of the functional approach.

EDIT: A bit unrelated to this exact question, but I’ve given my point of view on the usage of modules vs. functional API in this post,

2 Likes