I have a data set which looks like following:
Image_1 Image_2 Image_3
A B C
... ... ...
If image A is similar to B, it will be assigned with label 1, otherwise with label 0.
I first use pre-trained resnet18 to extract features for each RGB image and get a 1000 dimensional vector. Then I build a deep network and use triplet as loss function to train model, there is a part of my code:
class Network(torch.nn.Module):
def __init__(self, n_feature = 1000, n_hidden_1,n_output = 10):
super(Network, self).__init__()
self.net = torch.nn.Sequential(
torch.nn.Linear(n_feature, n_hidden_1),
torch.nn.BatchNorm1d(n_hidden_1),
torch.nn.ReLU(),
torch.nn.Linear(n_hidden_1, n_output)
)
def forward(self, x):
x = self.net(x)
return x
Training step:
for step, (batch_anchor, batch_positive, batch_negative )in enumerate(train_loader):
optimizer.zero_grad()
anchor_out = model(batch_anchor)
positive_out = model(batch_positive)
negative_out = model(batch_negative)
loss = loss_func(anchor_out, positive_out, negative_out)
loss.backward()
optimizer.step()
where I define loss function and optimiser with:
optimizer = optim.Adam(model.parameters(), lr=0.002)
loss_func = torch.nn.TripletMarginLoss()
After the training process is done, I test this network with validation set:
with torch.no_grad():
anchor_out_val = model(val_data_anchor).numpy()
positive_out_val = model(val_data_positive).numpy()
negative_out_val = model(val_data_negative).numpy()
Now I use L2 Norm to measure similarity and assign labels, this works very well in validation set and I got accuracy 80% measured by accuracy_score from sklearn. But when I try it with test set, I only get 50% accuracy. Maybe someone could tell me why? Is the metric to measure similarity not good? or maybe it is the problem from network?