Why is the pytorch Loss base class protected?

See torch.nn.modules.loss — PyTorch 1.8.1 documentation where classes _Loss and _WeightedLoss are protected and not getting imported in __init__.

What is the reason for this? For users who want to their own loss class it would be useful to both make use of the existing base classes and use them to better document the purpose of their own implementation as identifying them e.g. as a weighted loss.

The main reason for having the _Loss class is for the backward-compatibility of the reduction method.
There is nothing special about a loss compared to other nn.Module classes and I personally would think that it is good that no-one gets the idea of doing isinstance(l, Loss) anywhere because that would preclude passing in a nn.Module that does some processing and then calls loss.

Best regards

Thomas

I really cannot follow that argument - the ability to subclass opens the possibility to re-use code already there and is normally not something where checking with isinstance is a concern. ALL concrete implementations of loss classes can be subclassed anyways and the same goes for other module classes, but making e.g. _WeightedLoss protected just means that nobody can easily re-use the (reduction) code in that base class. My question was: why would that be a bad thing?
Not allowing it forces people to copy/paste the code or reimplement something comparable.

The idea here is that for any reasonably modern code, the re-usable part would be exactly self.reduction = reduction.
What I am trying to suggest is exactly this: New code should not provide the deprecated arguments size_average and reduction, so re-using the translation of them to then set self.reduction = … should not be terribly interesting.
When PyTorch finally drops the support for the deprecated arguments, _Loss might go away. (Yeah, and I readily admit that there is no reason why CTCLoss should subclass _Loss when adopting this line of reasoning.)

But maybe I am not understanding this right. Given the current constraints, what would you be copy-pasting to make your code work better?

Best regards

Thomas

1 Like

Sorry, my mistake, I misunderstood what the legacy reduction issue is all about.
Totally makes sense to me now!