Hi everyone, recently I started to read about Siamese Nets and I wanted to try this type of model on a gender recognition task. My data is a .csv dataset containing ~3000 n-vectors of audio features (n=20). The loss function I’m using is the Contrastive Loss. Here is my model:
class ContrastiveLoss(torch.nn.Module): """ Contrastive loss function. Based on: http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf """ def __init__(self, margin=1.0): super(ContrastiveLoss, self).__init__() self.margin = margin def forward(self, output1, output2, label): euclidean_distance = F.pairwise_distance(output1, output2) loss_contrastive = torch.mean((1-label) * torch.pow(euclidean_distance.double(), 2) + (label) * torch.pow(torch.clamp(self.margin - euclidean_distance.double(), min=0.0), 2)) return loss_contrastive class SiameseMLP(nn.Module): def __init__(self): super(SiameseMLP, self).__init__() self.layers = nn.Sequential( nn.Linear(20, 256), nn.ReLU(), nn.Dropout(p=0.1), nn.Linear(256, 256), nn.ReLU(), nn.Dropout(p=0.1), nn.Linear(256, 256), nn.ReLU(), nn.Dropout(p=0.1), nn.Linear(256, 2) ) def forward_once(self, x): x = x.view(-1, 20) x = self.layers(x) return x def forward(self, x_1, x_2): y_1 = self.forward_once(x_1) y_2 = self.forward_once(x_2) return y_1, y_2
I started to do some tests and for now I can’t reach a good value for the training loss, keeping it around ~0.33. When I calculate the dissimilarity between pairs, I get random values so I think the model is not learning completely. Do you have any suggest? Thanks.