I’m trying to use a convolutional auto-encoder to reconstruct some vectors (not images).
In order to assess whether it’s working I’d like to plot the cosine similarity between the input data and the reconstructed vector. What I’m doing right now is this:
model.eval()
with torch.no_grad():
for data in loader_val: # Validation dataset
# CAE
data, noise = data # Use data w/ noise as input model
data = data.unsqueeze(1).float()
noise = noise.unsqueeze(1).float()
out = model(noise)
loss = criterion(out, data)
# Cosine similarity
cos = CosineSimilarity()
cosine_sims.append(cos(data, out))
I wanted to visually see how it changes after each epoch and so I used the following commands to plot it:
plt.plot([np.mean(np.abs(el.numpy())) for el in cosine_sims])
The problem is that the values start at 0.9 and they just fluctuate (not decreasing or increasing). Is there something wrong in the way I use CosineSimilarity?
The Euclidean distance does not necessarily correlate with the angle (cosine similarity).
Imagine a point cloud near the origin of a coordinate system. While the MSELoss could be low, the cosine similarity might be high between certain points.
Would it be possible to use the CosineSimilarity as the criterion to train the model?
I’ve being thinking about this for a while now. I guess it’s possible but I don’t think I understand how to actually use it. The CosineSimilarity returns a number for each pair of vectors, and not a real loss. Should I take the mean or should I add it to MSELoss? Thanks.
Can a loss function be like:-
Total_loss = MSEloss()+CosineSimilarity()
I think this will be better measure.
Build your custom CosineSimilarityLoss if possible.
But I cannot simply add (or subtract) the cosine similarity to the loss function, since the former is not just a real number. Do you recommend taking the mean?