Question about concatenate the FC layer features of each model and use then train the concatenated features with FC layer

1

Untitled%20Diagram
I have 2 different models as shown above and I want to take features from FC layer of each model and concatenate them and pass it through another FC layer and train them. The question I have is if I should have 3 loss functions (one loss function for each network ) or one loss function at the end of the whole network. The concern I have about using one loss function is while backward propagation does the network choose to change the weights of only one network because it might lead to more decrease in loss

2

I think you should try. Try a lot, select one.

I have the same models as shown above, but I don’t know how to extract the FC layers, could you please help me with that?