RNN's and imbalanced data

[J_Johnson]

Standard accuracy scores mean very little with a highly unbalanced dataset.

If you have an 80:20 distribution, we can get 80% accuracy with standard scoring methods by just writing a few lines of code to always return the 80% class.

Here is an accuracy method which allows you to get class specific accuracy, or a mean of each class accuracy:

How do I get the accuracy of an unbalanced dataset or segmentation task?

This post is to define a Class Weighted Accuracy function(WCA). As questions related to this get asked often, I thought it might help people to post a tool torchers can use and reference here. Why? Take, for example, a classification dataset of kittens and puppies with a ratio of 0.2:0.8 kittens to puppies. A model trained on this dataset might show an overall accuracy of 80% by just predicting “puppies” every time. A WCA function resolves this problem by checking the accuracy of each class s…

Regarding training on an unbalanced dataset, loss functions allow you to pass in a weight(for binary classes) or pos_weight(for multi-class) argument. These will in turn increase the loss ratio for the minority classes and/or decrease the loss ratio for the majority classes.