Custom Loss Function for Unequal Weighted Multiple-Output Node Regression

1

I’m trying to build a regression network that has 16 outputs with one of the 16 outputs weighted 3 times as high (or X times as high in the general case) for loss purposes as the other 15 outputs. I have built a network that works for the 16 outputs when they are all equal weighted, but how would I go about up-weighting one of the outputs above the others? I feel like there should be a simple way of doing this that I’m not thinking of. Thanks for the help and ideas in advance!

So basically I need to create a custom loss function that takes the MSE of (predictions, targets) where the loss with respect to target_1 is X times the other targets.

Is there a way to alter the top level MSE_loss function to do this like this? :

 class MSELoss(_Loss):

    def __init__(self, size_average=True, reduce=True):
        super(MSELoss, self).__init__(size_average, reduce)

    def forward(self, input, target):
        _assert_no_grad(target)

        #Weight the first item in the input list and , target list or accept a variable to choose which item

        return F.mse_loss(input, target, size_average=self.size_average, reduce=self.reduce)
1 Like
2

Would this work?

batch_size = 2
model = nn.Linear(20, 16)
x = torch.randn(batch_size, 20)
y = torch.randn(batch_size, 16)
criterion = nn.MSELoss(reduce=False)
weight = torch.ones(16)
weight[1] = 3

output = model(x)
loss = criterion(output, y)
loss = loss * weight
loss.mean().backward()
6 Likes
3

Maybe ! let me see if that works in my code. Thanks for the reply. I was hoping it was something nice and simple like a vector of weights

4

I figured it out. Here is the working code for how to do this in the fast.ai library which is what I use on top of pytorch. m.crit is set by default in fast.ai for regression to F.mse_loss just for your reference

def weighted_mse_loss(input,target):
    #alpha of 0.5 means half weight goes to first, remaining half split by remaining 15
    weights = Variable(torch.Tensor([0.5,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15])).cuda()  
    pct_var = (input-target)**2
    out = pct_var * weights.expand_as(target)
    loss = out.mean() 
    return loss
1 Like