I have an output vector and a ground truth vector. I want to minimize the cosine distance between them but i have a constraint - i want that the output vector will have a l2-norm of 1 so i have created the following custom loss:
Outputs is a batch of size 32 that each is a vector of 60 dims.
So i am trying to minimize the cosine distance + |1-norm(output)|
But when i add the L1Loss the network does not learn anything and without the L1Loss i get accuracy of 98%.
So i guess that i am not using the L1 loss appropriately.
Can you please advise how can i solve it?
It looks like you’re mixing up L1 and L2 losses and norms both in your description and code (although it’s a bit hard to tell since the variable names are incosistent).
If you want your vectors to be L2-normalized, the corresponding loss is MSELoss.
If you want your vectors to be L1-normalized then the p in output1.norm should be 1.