When to use Module based loss functions vs functional loss functions

1

When would we choose to use the Module forms of the loss functions rather than the functional forms? For example, torch.nn.MSELoss over torch.nn.functional.mse_loss. I just realised that I’ve never used the Module forms, and I’m wondering whether I’m missing a useful trick or something? It seems to me that loss is something we would almost always want to calculate outside of the model, so it doesn’t make intuitive sense for it to be a Module.

2

I’m usually creating the criterion as a module in case I want to store some internal states, e.g. weight, a specific reduction etc.
Also, I would say it basically depends on your coding style and the use case you are working with.
E.g. if you are reusing the criterion in multiple places (e.g. GAN training) and would like to experiment with different loss functions as well as reductions, a single instance might be more convenient to use instead of the functional approach.

EDIT: A bit unrelated to this exact question, but I’ve given my point of view on the usage of modules vs. functional API in this post,

2 Likes
3

Thanks, that’s in line with how I’m thinking about it now. It’s good to know I wasn’t missing anything obvious.