As the title suggest the output of my model is around 0.5, hinting at uncertainty in predicting the class.
To give a little more information about my model:
It is designed to classify a pair of proteins as interacts (1) or doesn’t interact (0). The model consists of a graph neural network at the start, followed by an attention based feature fuser, and a classifier.
The code is:
class GAT_Model(nn.Module):
def init(self, num_features_pro, classifier_hidden_dim, input_dim, dropout, num_heads):
super(GAT_Model, self).init()
self.gat_layer = GATLayer(num_features_pro, input_dim, dropout, num_heads)
self.aff = AFF(input_dim)
self.classifier = MLPClassifier(input_dim, classifier_hidden_dim, dropout)def forward(self, protA_data, protB_data):
x1, x2 = self.gat_layer(protA_data, protB_data)
x, weights = self.aff(x1, x2)
output = self.classifier(x)
return output, weights
The code for my classifier module is also given below:
class MLPClassifier(nn.Module):
def init(self, input_dim, classifier_hidden_dim, dropout):
super(MLPClassifier, self).init()
self.fc1 = nn.Linear(input_dim, classifier_hidden_dim)
self.relu = nn.ReLU()
self.leaky_relu = nn.LeakyReLU(0.01)
self.dropout = nn.Dropout(dropout)
self.fc2 = nn.Linear(classifier_hidden_dim, 32)
self.fc3 = nn.Linear(32, 32)
self.fc4 = nn.Linear(32, 32)
self.output = nn.Linear(32, 1)
def forward(self, x):
x = torch.mean(x, dim=0)
x = x.unsqueeze(0)
#print("input to classifier:", x)
x = self.fc1(x) # layer 1
x = self.leaky_relu(x)
x = self.dropout(x)
x = self.fc2(x) # layer 2
x = self.leaky_relu(x)
x = self.dropout(x)
x = self.fc3(x) # layer 3
x = self.leaky_relu(x)
x = self.dropout(x)
x = self.fc4(x) # layer 4
x = self.relu(x)
x = self.dropout(x)
x = self.output(x)
#print("single output:", x)
x = torch.sigmoid(x)
return x
What can I do to improve the model. Predicted values for my validation set is all 0.522 for some reason, some of my training outputs are also given below (this is for the first epoch)
predicted labels train: tensor(0.5085, grad_fn=)
true labels train: tensor(1.)
predicted labels train: tensor(0.4979, grad_fn=)
true labels train: tensor(1.)
predicted labels train: tensor(0.5188, grad_fn=)
true labels train: tensor(0.)
predicted labels train: tensor(0.5247, grad_fn=)
true labels train: tensor(0.)
predicted labels train: tensor(0.5191, grad_fn=)
true labels train: tensor(1.)
predicted labels train: tensor(0.5144, grad_fn=)
true labels train: tensor(0.)
predicted labels train: tensor(0.5043, grad_fn=)
true labels train: tensor(0.)
predicted labels train: tensor(0.5380, grad_fn=)
true labels train: tensor(0.)
predicted labels train: tensor(0.4969, grad_fn=)
true labels train: tensor(1.)
predicted labels train: tensor(0.5032, grad_fn=)
true labels train: tensor(0.)
predicted labels train: tensor(0.5063, grad_fn=)
true labels train: tensor(1.)
predicted labels train: tensor(0.5054, grad_fn=)
true labels train: tensor(0.)