Okay, so I implemented your code that you referenced here for balancing datasets with WeightedRandomSampler and it seems like it worked?
target train: 6361/339
batch index 0, 0/1: 25/25
batch index 1, 0/1: 24/26
batch index 2, 0/1: 28/22
batch index 3, 0/1: 23/27
batch index 4, 0/1: 22/28
batch index 5, 0/1: 29/21
batch index 6, 0/1: 21/29
batch index 7, 0/1: 17/33
batch index 8, 0/1: 20/30
batch index 9, 0/1: 19/31
batch index 10, 0/1: 25/25
batch index 11, 0/1: 29/21
batch index 12, 0/1: 23/27
batch index 13, 0/1: 27/23
batch index 14, 0/1: 26/24
batch index 15, 0/1: 28/22
batch index 16, 0/1: 24/26
batch index 17, 0/1: 24/26
batch index 18, 0/1: 22/28
batch index 19, 0/1: 26/24
batch index 20, 0/1: 22/28
batch index 21, 0/1: 23/27
batch index 22, 0/1: 28/22
batch index 23, 0/1: 26/24
batch index 24, 0/1: 27/23
batch index 25, 0/1: 28/22
batch index 26, 0/1: 22/28
batch index 27, 0/1: 22/28
batch index 28, 0/1: 21/29
batch index 29, 0/1: 23/27
batch index 30, 0/1: 27/23
batch index 31, 0/1: 19/31
batch index 32, 0/1: 26/24
batch index 33, 0/1: 30/20
batch index 34, 0/1: 22/28
batch index 35, 0/1: 21/29
batch index 36, 0/1: 23/27
batch index 37, 0/1: 28/22
batch index 38, 0/1: 29/21
batch index 39, 0/1: 27/23
batch index 40, 0/1: 25/25
batch index 41, 0/1: 28/22
batch index 42, 0/1: 22/28
batch index 43, 0/1: 25/25
batch index 44, 0/1: 29/21
batch index 45, 0/1: 20/30
batch index 46, 0/1: 24/26
batch index 47, 0/1: 29/21
batch index 48, 0/1: 27/23
batch index 49, 0/1: 28/22
batch index 50, 0/1: 24/26
batch index 51, 0/1: 26/24
batch index 52, 0/1: 24/26
batch index 53, 0/1: 24/26
batch index 54, 0/1: 25/25
batch index 55, 0/1: 26/24
batch index 56, 0/1: 25/25
batch index 57, 0/1: 25/25
batch index 58, 0/1: 30/20
batch index 59, 0/1: 26/24
batch index 60, 0/1: 24/26
batch index 61, 0/1: 22/28
batch index 62, 0/1: 24/26
batch index 63, 0/1: 25/25
batch index 64, 0/1: 30/20
batch index 65, 0/1: 31/19
batch index 66, 0/1: 20/30
batch index 67, 0/1: 26/24
batch index 68, 0/1: 27/23
batch index 69, 0/1: 20/30
batch index 70, 0/1: 28/22
batch index 71, 0/1: 25/25
batch index 72, 0/1: 28/22
batch index 73, 0/1: 28/22
batch index 74, 0/1: 26/24
batch index 75, 0/1: 25/25
batch index 76, 0/1: 23/27
batch index 77, 0/1: 27/23
batch index 78, 0/1: 27/23
batch index 79, 0/1: 29/21
batch index 80, 0/1: 19/31
batch index 81, 0/1: 28/22
batch index 82, 0/1: 26/24
batch index 83, 0/1: 22/28
batch index 84, 0/1: 26/24
batch index 85, 0/1: 23/27
batch index 86, 0/1: 25/25
batch index 87, 0/1: 29/21
batch index 88, 0/1: 19/31
batch index 89, 0/1: 26/24
batch index 90, 0/1: 25/25
batch index 91, 0/1: 27/23
batch index 92, 0/1: 26/24
batch index 93, 0/1: 30/20
batch index 94, 0/1: 24/26
batch index 95, 0/1: 22/28
batch index 96, 0/1: 24/26
batch index 97, 0/1: 21/29
batch index 98, 0/1: 30/20
batch index 99, 0/1: 25/25
batch index 100, 0/1: 28/22
batch index 101, 0/1: 24/26
batch index 102, 0/1: 24/26
batch index 103, 0/1: 28/22
batch index 104, 0/1: 25/25
batch index 105, 0/1: 22/28
batch index 106, 0/1: 23/27
batch index 107, 0/1: 25/25
batch index 108, 0/1: 28/22
batch index 109, 0/1: 26/24
batch index 110, 0/1: 27/23
batch index 111, 0/1: 27/23
batch index 112, 0/1: 33/17
batch index 113, 0/1: 26/24
batch index 114, 0/1: 27/23
batch index 115, 0/1: 29/21
batch index 116, 0/1: 25/25
batch index 117, 0/1: 28/22
batch index 118, 0/1: 22/28
batch index 119, 0/1: 23/27
batch index 120, 0/1: 25/25
batch index 121, 0/1: 17/33
batch index 122, 0/1: 28/22
batch index 123, 0/1: 24/26
batch index 124, 0/1: 22/28
batch index 125, 0/1: 23/27
batch index 126, 0/1: 26/24
batch index 127, 0/1: 26/24
batch index 128, 0/1: 24/26
batch index 129, 0/1: 27/23
batch index 130, 0/1: 22/28
batch index 131, 0/1: 23/27
batch index 132, 0/1: 27/23
batch index 133, 0/1: 25/25
However, when I run my models, it still simply outputs the majority class? Do you have any further thoughts?
train_model(baseline, epochs, learning_rate = 0.1)
Iteration: 200. Loss: 0.352510005235672. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 400. Loss: 0.29750311374664307. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 600. Loss: 0.4701906442642212. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 800. Loss: 0.3363111913204193. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 1000. Loss: 0.37919408082962036. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 1200. Loss: 0.3066049814224243. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 1400. Loss: 0.3903108239173889. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 1600. Loss: 0.36794665455818176. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 1800. Loss: 0.44550082087516785. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 2000. Loss: 0.3176323175430298. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 2200. Loss: 0.3094715476036072. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 2400. Loss: 0.3805839419364929. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
Iteration: 2600. Loss: 0.39165371656417847. Accuracy: 95.12121212121212
Sum of Predictions: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0])
I also tried using a weighted loss, but I got too confused and made some mistakes in constructing that, so I went with WeightedRandomSampler instead