A model with multiple outputs

1

I have to implement a Convolutional Neural Network, that takes a kinect image (1640480) and return a 1 x8 tensor predicting the class to which the object belongs and a 1 x 4 tensor, predicting the bounding box around the image, if its present.

Please help me how can I implement a suitable model to give two outputs and how to calculate loss and backpropagate in that case?
Also, i have just around 6000 training images, how can I achieve the best possible results with limited number of training images??

Some examples of such projects would be highly helpful.

10 Likes
2

Multiple outputs is pretty straightforward. Just return mutiple values in the forward() method of your net.

def forward(self, x):
    # Do your stuff here
    ...
    x1 = F.log_softmax(x) # class probabilities
    x2 = ... # bounding box calculation
    return x1, x2

Using these two outputs, you can define two different loss functions and just add them.

out1, out2 = model(data)
loss1 = criterion1(out1, target1)
loss2 = criterion2(out2, target2)
loss = loss1 + loss2
loss.backward()
44 Likes
3

please dont tag folks generically, it’s not helpful

4

Ok
I am sorry for that
Thanks

5

Thanks for your answer!

6

Hi, Smth
Actually , I still dont understand your idea about tag folks, does it mean the forward function from ptrblcks example sets tow return value or his loss function uses tow loss addition ？ can you explain it in detail ？ can you give a example code ？
Thank you very much

7

Hi, can we pack the returned tensors in list format? Such as:
list = [x1, x2]
return list

Or, we must return tensor class type in forward method?

1 Like
8

Yes, you can also return a list. Here is a small dummy example:

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(1, 1)
        
    def forward(self, x):
        y = self.fc1(x)
        return [y, x]

model = MyModel()
x = torch.randn(1, 1)
outputs = model(x)
print(outputs)
4 Likes
9

Thank you so much for your quick help!

10

Hey,
I was going through the keras documentation
https://keras.io/getting-started/functional-api-guide/#multi-input-and-multi-output-models
As you can see this is an example of multi-output multi input model.The thing in this example is that the auxiliary output has a loss function .This output is then further used in the model to compute the main_output.
How can I implement some thing like this in Pytorch

11

You could just re-write the forward pass in your model similar to this:

def forward(self, main_input, aux_input):
    main_input = self.embedding_1(main_input)
    main_input = self.lstm(main_input, (self.hidden, self.state))
    aux = self.aux_output(main_input)    

    aux_input = self.input_layer(aux_input)
    x = torch.cat((aux_input, main_input))
    x = F.relu(self.dense_1(x))
    x = F.relu(self.dense_2(x))
    x = F.relu(self.dense_3(x))
    x = self.main_output(x)

    return x, aux
3 Likes
12

Oh okay .Thanks a lot.
Can I also use two different loss functions one on auxiliary_output and one on main_output as shown in the keras link and then add them in the end.

13

Sure, that should also work!
Let me know, if you encounter any problems. :wink:

1 Like
14

I would like to do the same, but accumulate loss in a loop as my model’s forward function returns a list of outputs.
My question is, how do I initialize the “loss” variable so that I can accumulate in a loop?

15

If your model returns multiple outputs, you could calculate the loss separately for each output and then just accumulate it:

output1, output2, output3 = model(data)
loss1 = criterion1(output1, target1)
loss2 = criterion2(output2, target2)
loss3 = criterion3(output3, target3)
loss = loss1 + loss2 + loss3

# or
outputs = model(data)
loss = 0
for output, target, criterion in zip(outputs, targets, criteria):
      loss = loss + criterion(output, target)
3 Likes
17

Little confused about the accumulation “+”:
So when I write loss = loss1 + loss2, the network knows that I want loss1 to backpropagate through output1, and loss2 to go thru output2, the sequence matters, am I right?

2 Likes
18

The addition is treated as any other operation in your model, so Autograd will calculate the gradients based on the derivative of the loss accumulation.

If the outputs are independent, your assumption is correct.

6 Likes
19

hey , i have a problem with “Assertion `cur_target >= 0 && cur_target < n_classes’ failed. at /pytorch/aten/src/THNN/generic/ClassNLLCriterion.c:97” which is still persisting even i tried many solutions found on internet.

20

Could you print the target values of your current batch inside the training loop and check, if they are all in the range [0, nb_classes-1]?
If you run the code with num_workers=0, the last printed target should be the one causing this error.

21

It start counting from 1.