Hi, i’m trying to do multiobjective optimization with using deep learning model.I would like to take the predictions for each task from a deep learning model with more than two dimensional outputs and put them into separate loss functions for consideration, but I don’t know how to do it.
I have been able to implement this to the point where I can extract predictions for each task from a deep learning model with more than two dimensional outputs, so I would like to know how I can properly use the loss function.
code is below
criterion = nn.MSELoss()
for epoch in (range(num_epochs)):
train_running_loss = 0.0
for torch_comp, torch_comp_wt, task_num, y in train_loader:
torch_comp = torch_comp.to(device)
torch_comp_wt = torch_comp_wt.to(device)
task_num = task_num.reshape(-1, 1).to(device)
y = torch.Tensor(y).to(device)
y_pred = model(torch_comp, torch_comp_wt) #(256, 2)
y_pred = torch.take_along_dim(y_pred, task_num, dim=1) # get Predicted value per task
loss = (criterion(y_pred, y.unsqueeze(1))) # I want to change here!!!!!!
What you are actually trying to do in deep learning is called multi-task learning. As you mentioned, you get multiple prediction outputs based on different loss functions. For this you first have to define an architecture. The easiest and most simplest one is based on Caruana from the 90s .
One architecture might look like this where you assume two inputs based on
x and three outputs based on
y. Afterwards it could look somewhat like this
self.dense1 = torch.nn.Conv2d(3,32, kernel_size=3, stride=1, padding=1)
self.flatten = torch.nn.Flatten()
# resnet18_fe provide (512, ) features x 3 images concatenated together
self.fc1 = nn.Linear(50176,4) # 10 is the max value of each label
self.fc2 = nn.Linear(50176,4)
self.fc3 = nn.Linear(50176, 4)
def forward(self, x):
y1 = self.dense1(x)
y2 = self.dense1(x)
y1, y2 = cross_stich_network(y1,y2)
y1 = self.flatten(y1)
y2 = self.flatten(y2)
y = torch.concat([y1, y2], dim=-1)
to calculate the loss you can simply add the losses for each criteria such that you something like this
total_loss = criterion(y_pred, label) + criterion(y_pred, label) + criterion(y_pred, label)
I hope this helps.
 Multitask Learning | SpringerLink