Multiple Criterions Error: unsupported operand type for '+' (v0.2)

I want to compute multiple losses (as shown in this question ), but doing it the same way creates an error (pytorch v0.2):

Traceback (most recent call last):
File "tmp.py", line 159, in <module>
loss = criterion1 + criterion2
TypeError: unsupported operand type(s) for +: 'MSELoss' and 'L1Loss'

My code (selected snippets) is the fallowing:

criterion1  = nn.MSELoss(size_average=False).cuda()
criterion2  = nn.L1Loss(size_average=False).cuda()
loss = criterion1 + criterion2
train(epochs)

and

def train(epochs):   
    epoch = 1
    while epoch <= epochs:
        for batch_idx, (data, _ ) in enumerate(train_loader):
            data = Variable(data.type(torch.FloatTensor).cuda())
            optimizer.zero_grad()
            output, y, z = model(data)
            loss1 = criterion1(output, data)
            loss2 = criterion2(z, y)
            loss.backward()
            optimizer.step()

with my Net:

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(100, 100)
        self.fc1.weight.data = torch.from_numpy(A).type(torch.FloatTensor)
        [...]
        
    def forward(self, x):
        y = self.fc1(x)
        x = self.rl1(self.fc2(y))
        [...]
        x_est = self.fc5(x)
        z_est = self.fc1(x_est)
        return x_est, y, z_est

You should not add the nn.Criterion together, but instead the losses.
So you should do

criterion1  = nn.MSELoss(size_average=False).cuda()
criterion2  = nn.L1Loss(size_average=False).cuda()

output = model(input)

loss = criterion1(output, target) + criterion2(output, target)

That being said, you can create a function that simplifies your life for that

class MyCriterion(nn.Module):
    def __init__(self, size_average=False):
        super(MyCriterion, self).__init__()
        self.criterion1  = nn.MSELoss(size_average=size_average)
        self.criterion2  = nn.L1Loss(size_average=size_average)
    def forward(self, input, target):
        return self.criterion1(input, target) + self.criterion2(input, target)

criterion = MyCriterion().cuda()
2 Likes

Thanks! I was a little bit confused, I guess…