Hello guys, I would like to implement below loss function which is a weighted mean square loss function:
nn.MSELoss to achieve to my mentioned loss function?
You can probably use a combination of tensor operations to compute your loss.
For example
def mse_loss(input, target):
return torch.sum((input - target) ** 2)
def weighted_mse_loss(input, target, weight):
return torch.sum(weight * (input - target) ** 2)
What I have understood from your above snippet is to create a nn.Module and write your above code in its forward function. Am I right? Because I would like to use autograd for backprop!
You don’t need to write a nn.Module for that, a simple function is enough.
Note that backpropagation is handled by Variables, and not by nn.Module.
So this is perfectly fine
def pow(input, n):
return input ** n
t = Variable(torch.rand(1), requires_grad=True)
t2 = pow(input, 2)
t6 = pow(t2, 3)
t6.backward()
Thank you. 
I appreciate your response!
Were you able to solve this? When i try to set this function as my criterion i get an error saying the function requires the 3 inputs which obviously aren’t calculated yet until training. Maybe this is why it needs to go in nn.module ?
This is the ugly hack i created that works for this problem. 16 outputs with the first output being weighted 8/16ths and the remaining outputs weighted 0.5/15.
I’m sure there’s a better way to do this but if you’re in a hurry this works.
def weighted_mse_loss(input,target):
#alpha of 0.5 means half weight goes to first, remaining half split by remaining 15
weights = Variable(torch.Tensor([0.5,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15])).cuda()
pct_var = (input-target)**2
out = pct_var * weights.expand_as(target)
loss = out.mean()
return loss
your question is somehow irrelevant.
Hi,
It’s not a big deal but why isn’t weighting provided in default implementation since all (atleast most) other loss functions have it?
Actually,
def weighted_mse_loss(input, target, weight):
return (weight * (input - target) ** 2).mean()
is slightly better, since the sum is some big. Taking the average is exactly the original way that nn.MSELoss does.
I think it is better divided by the sum of weight instead of taking average cause it is how the weighted cross entropy loss implemented
def weighted_mse_loss(input, target, weight):
return (weight * (input - target) ** 2).sum() / weight.sum()