How to implement intermediate supervision in PyTorch?

1

Does anybody know how to add intermediate supervision in PyTorch, which means the network produces multiple predictions and is a common technique used in Googlenet and stacked hourglass network?

1 Like
2

Also having this issue :slight_smile:

3

Me, too. Stacking multiple stage networks while keeping the intermediate output of each stage is a problem. It seems that for loop couldn’t be used.

4

I don’t know if I fully understand the problem. You can just create a list, and append all the intermediate states you want to it.

1 Like
5

There’s no reason a for loop couldn’t be used. Assuming skip_layers and hourglass_layers contain the necessary modules:

skips = []
for skip_layer, hourglass_layer in zip(skip_layers, hourglass_layers):
    skips.append(skip_layer(input))
    input = hourglass_layer(input)

Do that for the first half of the hourglass, then another for loop for the second half where you combine the output in skips and use that as input for the rest of the “stem” layers.

2 Likes
7

Thanks a lot for the hint. But I’m still a little bit confused about how to implement it ( sorry a newbie here ). It would be great if you would share a full hourglass model here for reference.