How to do multiobjective optimization?

1

Hi, i’m trying to do multiobjective optimization with using deep learning model.I would like to take the predictions for each task from a deep learning model with more than two dimensional outputs and put them into separate loss functions for consideration, but I don’t know how to do it.
I have been able to implement this to the point where I can extract predictions for each task from a deep learning model with more than two dimensional outputs, so I would like to know how I can properly use the loss function.

Thank you.

code is below

・
・
criterion = nn.MSELoss()
for epoch in (range(num_epochs)):
        model.train()
        train_running_loss = 0.0
        for torch_comp, torch_comp_wt, task_num, y in train_loader:
            torch_comp = torch_comp.to(device)
            torch_comp_wt = torch_comp_wt.to(device)
            task_num = task_num.reshape(-1, 1).to(device)
            y = torch.Tensor(y).to(device)
            

            optimizer.zero_grad()
            y_pred = model(torch_comp, torch_comp_wt) #(256, 2)
            # print(task_num)
            # print(y_pred[:5])
            y_pred = torch.take_along_dim(y_pred, task_num, dim=1) # get Predicted value per task

            loss = (criterion(y_pred, y.unsqueeze(1))) # I want to change here!!!!!!


            loss.backward()
            optimizer.step()
2

What you are actually trying to do in deep learning is called multi-task learning. As you mentioned, you get multiple prediction outputs based on different loss functions. For this you first have to define an architecture. The easiest and most simplest one is based on Caruana from the 90s [1].
One architecture might look like this where you assume two inputs based on x and three outputs based on y. Afterwards it could look somewhat like this

class Model(nn.Module):
    
    def __init__(self):
        super().__init__()
        
        self.dense1 = torch.nn.Conv2d(3,32, kernel_size=3, stride=1, padding=1)
        
        self.flatten = torch.nn.Flatten()
        
        # resnet18_fe provide (512, ) features x 3 images concatenated together 
        self.fc1 = nn.Linear(50176,4)  # 10 is the max value of each label
        self.fc2 = nn.Linear(50176,4)
        self.fc3 = nn.Linear(50176, 4)
        
    def forward(self, x):        
        y1 = self.dense1(x[0])
        y2 = self.dense1(x[1])
        
        y1, y2 = cross_stich_network(y1,y2)
        
        y1 = self.flatten(y1)
        y2 = self.flatten(y2)
        
        y = torch.concat([y1, y2], dim=-1)
        return [
            self.fc1(y),
            self.fc2(y),
            self.fc3(y),           
        ]

to calculate the loss you can simply add the losses for each criteria such that you something like this

total_loss = criterion(y_pred[0], label[0]) + criterion(y_pred[1], label[1]) + criterion(y_pred[2], label[2])

I hope this helps.

[1] Multitask Learning | SpringerLink

1 Like