I am using CosineEmbeddingLoss with a margin= 0.35. My loss just keep fluctuating until 500 epochs and it does not smoothen out.
Here are my parameters
Adam: Momentum = 0.9,Weight Decay = 0.01, LR = 0.000001.
Batch Size = 64
The network is a simple Auto Encoder.
