Possible to classify image based on its proximity to similar ones?

Hello everyone,

I’m looking for help in finding some reading material on the topic of classifying an image based on its proximity to a reference image. I’m working on a project just now where I try to separate cat & dog images based on their cosine similarities. I’m now getting an accuracy of about 98.5% on a small validation set by using Resnet18.

The way that it works (briefly) is that I compute embeddings for an unknown image and checks it cosine similarity against a bunch of dog & cat reference images. If the unknown image matches better with the dog reference images, I’ll classify the unknown image as a dog.

The closest method I can think of is e.g. identifying people via triplet loss. I also know there are fields such as image matching or image retrieval that tangent the subject.

I’m looking for resources or people willing to discuss the subject of classifying a data point in accordance to its proximity with other similar data points.