Non-legacy equivalent of ParallelCriterion in PyTorch

1

I’m porting some Lua/Torch code over to PyTorch, which used ParallelCriterion (to apply L1 loss separately to 2 different layers of a stacked network, applying final+intermediate supervision).

I see that the legacy nn API provided by PyTorch provides a ParallelCriterion implementation – but what is the “correct” / non-legacy way to do this?

Thanks for any help!

2

you simply do:

loss1 = criterion1(input1[:, 0])
loss2 = criterion2(input1[:, 1])
torch.cat(loss1, loss2, dim=0)

(an example of a two-criterion case is shown, if you have a lot of criterion, just do a for loop)

3

Thanks for your reply!

I have tried this, but when I call backward() on the resulting combined loss, I receive the following error:

“grad can be implicitly created only for scalar outputs”

4

well, yes. your final output Variable is not a scalar if you cat two losses.

Maybe your intention is instead:

loss1 = criterion1(input1[:, 0])
loss2 = criterion2(input1[:, 1])
loss = loss1 + loss2
5

Thanks, that was it!