Error Analysis on Binary classification

1

Hi

Suppose you have a NN model which predicts a probability (Sigmoid in the last layer and BCELoss as a loss function), and the target column has the values True or False, a you use roc_auc as a accuracy metric.

  1. How can you perform an Error Analysis? I mean … take a look at the missclassified samples
  2. How can I know if a sample was missclassified? I don’t have any probability threshold to classify a sample as target=True or target=False

Best regards

2

You could use the ROC to pick a threshold value and create the predictions using this threshold.
Once you have the predictions you could check all misclassified samples, compute a confusion matrix etc. for further investigation.