Loss for multiple output in multitask CNN

[puzzledqs (Shi Qiu, Facebook)]

I think the first two pieces of code should work fine and have the same net effect.
The last one seems incorrect as it would duplicate the backward computation.
You can refer to post below for more explanations

How to combine multiple criterions to a loss function?

Yeah, you can optimize a variable number of losses without a problem. I’ve had an application like that, and I just used total_loss = sum(losses) where losses is a list of all costs. Calling .backward() on that should do it. Note that you can’t expect torch.sum to work with lists - it’s a method for Tensors. As I pointed out above you can use sum Python builtin (it will just call the + operator on all the elements, effectively adding up all the losses into a single one). Iterating over a batch …