Piecewise losses in pytorch

1

Hi Pytorch,
I’m trying to implement a custom piecewise loss function in pytorch. Specifically the reverse huber loss with an adaptive threshold (Loss = |x| if |x| <c, x^2+c^2/2*c otherwise) . There doesn’t seem to be a great way to do this. Below is my code
`adiff = torch.abs(output-target)
batch_max = 0.2torch.max(adiff).data[0]
t1_mask = adiff.le(batch_max).float()
t2_mask = adiff.gt(batch_max).float()
t1 = adifft1_mask
t2 = (adiffadiff+batch_maxbatch_max)/(2batch_max)
t2 = t2t2_mask
return (torch.sum(t1)+torch.sum(t2))/torch.numel(output.data)``

not the most straightforward. I’m sort of struggling to figure out how I can implement a piecewise loss function easily without using this cumbersome masking thing (in fact, I’m struggling to determine if this is actually correct or not). Can someone help, point me towards an example, or otherwise provide guidance?

2

If that does what you want, it probably is correct.
An easier way to express it would be something like

adiff = torch.abs(output-target)
batch_max = 0.2*torch.max(adiff).data[0]
sqdiff = (adiff*adiff+batch_max*batch_max)/(2*batch_max)
return adiff.clamp(max=batch_max)+(sqdiff-1).clamp(min=batch_max)

Note though that clamp is not differentiable w.r.t. the bounds, but your definition of batch_max isn’t, either. It probably does not matter much.

Best regards

Thomas

3

thanks. its a per-batch statistic. I’m open to other suggestions if you’re worried it could be a problem, but at the very least autograd is able to differentiate it. btw, I ended up running torch.mean() on that expression prior to returning it.