I want to train a convolutional network with triplet loss for getting accurate embeddings and then feed the embeddings to a dense network and train the dense network with other loss.
Now What should be the best practice to do this in pytorch.
i) Make a common network and freeze different part at a time with each loss for training.
ii) Define two different network blocks and train them with each loss.
And if ii) then while training second block, how should i manage with the first block gradient update, With torch.no_grad() while feeding in first block, or detach the output of first block.
Or any other idea to deal with this