I have a data set which looks like following:
Image_1 Image_2 Image_3 A B C ... ... ...
If image A is similar to B, it will be assigned with label 1, otherwise with label 0.
I first use pre-trained resnet18 to extract features for each RGB image and get a 1000 dimensional vector. Then I build a deep network and use triplet as loss function to train model, there is a part of my code:
class Network(torch.nn.Module): def __init__(self, n_feature = 1000, n_hidden_1,n_output = 10): super(Network, self).__init__() self.net = torch.nn.Sequential( torch.nn.Linear(n_feature, n_hidden_1), torch.nn.BatchNorm1d(n_hidden_1), torch.nn.ReLU(), torch.nn.Linear(n_hidden_1, n_output) ) def forward(self, x): x = self.net(x) return x
for step, (batch_anchor, batch_positive, batch_negative )in enumerate(train_loader): optimizer.zero_grad() anchor_out = model(batch_anchor) positive_out = model(batch_positive) negative_out = model(batch_negative) loss = loss_func(anchor_out, positive_out, negative_out) loss.backward() optimizer.step()
where I define loss function and optimiser with:
optimizer = optim.Adam(model.parameters(), lr=0.002) loss_func = torch.nn.TripletMarginLoss()
After the training process is done, I test this network with validation set:
with torch.no_grad(): anchor_out_val = model(val_data_anchor).numpy() positive_out_val = model(val_data_positive).numpy() negative_out_val = model(val_data_negative).numpy()
Now I use L2 Norm to measure similarity and assign labels, this works very well in validation set and I got accuracy 80% measured by accuracy_score from sklearn. But when I try it with test set, I only get 50% accuracy. Maybe someone could tell me why? Is the metric to measure similarity not good? or maybe it is the problem from network?