How to apply gradient to tuneable gamma parameter in loss function

[Tony-Y]

How to learn the weights between two losses?

In order to avoid numerical instability, we should use a variable change: eta = log(sigma) The new variable eta can be defined within (-oo, +oo). [multitask_loss] Sample code: import torch import torch.nn as nn import torch.optim as optim class MultiTaskLoss(nn.Module): def __init__(self, model, loss_fn, eta): super(MultiTaskLoss, self).__init__() self.model = model self.loss_fn = loss_fn self.eta = nn.Parameter(torch.Tensor(eta)) def forward(self,…

You can find an PyTorch implementation for such a loss in this thread.