How to plot roc curve for custom NN classifier

Hi!

I have a custom classifier (binary class), and would like to plot ROC curve. I understand that I have to feed the X test values and the probabilities that either value would belong to class 1 or 0. This is where I’m stuck: all the codes that I have seen use libraries such as Scikit Learn, which have their own methods for this. How do I calculate these probabilities using my own classifier?

The code to my classifier is:


class NNClassifier(torch.nn.Module):
    def __init__(self):
        super(NNClassifier, self).__init__()
        self.linear = torch.nn.Linear(672, 1)
        self.layer_1 = nn.Linear(672, 300) 
        self.layer_2 = nn.Linear(300, 300)
        self.layer_out = nn.Linear(300, 1) 
        
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(p=0.1)
        self.batchnorm1 = nn.BatchNorm1d(300)
        self.batchnorm2 = nn.BatchNorm1d(300)
        
        for m in self.modules():
            if m.__class__ == nn.Conv2d or m.__class__ == nn.Linear:
                init.xavier_normal_(m.weight.data)
                m.bias.data.fill_(0)
                
    def forward(self, inputs):
        x = self.relu(self.layer_1(inputs))
        x = self.batchnorm1(x)
        x = self.relu(self.layer_2(x))
        x = self.batchnorm2(x)
        x = self.dropout(x)
        x = self.layer_out(x)
        
        return x

Any help is appreciated!

Hi,

so you should be getting outputs of shape batch_size x 1 from your model. If you loop over your test set and collect them and the labels into two lists as discussed in this thread and concatenate (torch.cat) to get prediction scores and labels for the entire dataset.

You can then either feed the result to SKLearn or do this manually as we did in Chapter 14 or our book by sorting the predictions and then getting tp / fp per threshold with broadcasting (cell 4) and then doing the numerical integration using the trapezoidal rule (cell 5).

Best regards

Thomas

1 Like