Hello, I am a training multiclass classifier. I want to calculate the F1 score on the whole dataset. earlier I calculated f1 for each batch using scikit-learn and I averaged f1 scores on batches, but it seems that it’s not the correct solution, it will be close to real f1 but it’s still not the exact value. the dataset is about 5GB to 10GB. How can I calculate the f1 score on the whole dataset?
You technically only need the y_pred, y_true and labels to calculate the F1 score as per scikit-learn's F1 score documentation. You really don’t need anything else from your 5-10 GB dataset.
Perhaps I’ve misunderstood what you’re trying to do, in which case, could you give a bit more detail on what you’re trying to do?
solution is to use Pytorch lightning
1 Like