Hello everyone. Sorry if it is a stupid question.
I am trying to train a siamese network for speaker identification.
For this task I am trying to train a small CNN with triplet margin loss to generate embeddings to distinguish each speaker.
Before I try to use some hard triplet mining, I want to test a simple batch all triplet approach.
For this I must pass all triplets in my minibatch through the network, and average their losses by the number of non zero losses.
For this I have a doubt. There is any difference between this two segments of code?
loss = criterion(anchor_feats, pos_feats, neg_feats)
loss = loss[loss!=0].mean()
loss.backward()
optimizer.step()
loss = criterion(anchor_feats, pos_feats, neg_feats)
loss = loss.sum()/((loss!=0).sum())
loss.backward()
optimizer.step()
Thank you very much.