See torch.nn.modules.loss — PyTorch 1.8.1 documentation where classes
_WeightedLoss are protected and not getting imported in
What is the reason for this? For users who want to their own loss class it would be useful to both make use of the existing base classes and use them to better document the purpose of their own implementation as identifying them e.g. as a weighted loss.
The main reason for having the
_Loss class is for the backward-compatibility of the reduction method.
There is nothing special about a loss compared to other
nn.Module classes and I personally would think that it is good that no-one gets the idea of doing
isinstance(l, Loss) anywhere because that would preclude passing in a
nn.Module that does some processing and then calls loss.
I really cannot follow that argument - the ability to subclass opens the possibility to re-use code already there and is normally not something where checking with isinstance is a concern. ALL concrete implementations of loss classes can be subclassed anyways and the same goes for other module classes, but making e.g. _WeightedLoss protected just means that nobody can easily re-use the (reduction) code in that base class. My question was: why would that be a bad thing?
Not allowing it forces people to copy/paste the code or reimplement something comparable.
The idea here is that for any reasonably modern code, the re-usable part would be exactly
self.reduction = reduction.
What I am trying to suggest is exactly this: New code should not provide the deprecated arguments
reduction, so re-using the translation of them to then set
self.reduction = … should not be terribly interesting.
When PyTorch finally drops the support for the deprecated arguments,
_Loss might go away. (Yeah, and I readily admit that there is no reason why
CTCLoss should subclass
_Loss when adopting this line of reasoning.)
But maybe I am not understanding this right. Given the current constraints, what would you be copy-pasting to make your code work better?
Sorry, my mistake, I misunderstood what the legacy reduction issue is all about.
Totally makes sense to me now!