Piecewise losses in pytorch

Hi Pytorch,
I’m trying to implement a custom piecewise loss function in pytorch. Specifically the reverse huber loss with an adaptive threshold (Loss = |x| if |x| <c, x^2+c^2/2*c otherwise) . There doesn’t seem to be a great way to do this. Below is my code
`adiff = torch.abs(output-target)
batch_max = 0.2torch.max(adiff).data[0]
t1_mask = adiff.le(batch_max).float()
t2_mask = adiff.gt(batch_max).float()
t1 = adiff
t2 = (adiffadiff+batch_maxbatch_max)/(2batch_max)
t2 = t2
return (torch.sum(t1)+torch.sum(t2))/torch.numel(output.data)``

not the most straightforward. I’m sort of struggling to figure out how I can implement a piecewise loss function easily without using this cumbersome masking thing (in fact, I’m struggling to determine if this is actually correct or not). Can someone help, point me towards an example, or otherwise provide guidance?

If that does what you want, it probably is correct.
An easier way to express it would be something like

adiff = torch.abs(output-target)
batch_max = 0.2*torch.max(adiff).data[0]
sqdiff = (adiff*adiff+batch_max*batch_max)/(2*batch_max)
return adiff.clamp(max=batch_max)+(sqdiff-1).clamp(min=batch_max)

Note though that clamp is not differentiable w.r.t. the bounds, but your definition of batch_max isn’t, either. It probably does not matter much.

Best regards


thanks. its a per-batch statistic. I’m open to other suggestions if you’re worried it could be a problem, but at the very least autograd is able to differentiate it. btw, I ended up running torch.mean() on that expression prior to returning it.