Hi friends,
I’ve noticed something strange during training a simple lenet model on mnist fashion:
The accuracy seems to not increase even thought the loss reaches as low as 0.01xx.
ain: (Epoch 45 of 100) [0665/0860] Time: 0.00665 (4.149) Loss: 0.40305 (0.235) Error: 0.88672 (0.887) Accuracy 0.11328125 [694/1967]
Train: (Epoch 45 of 100) [0666/0860] Time: 0.00632 (4.155) Loss: 0.38985 (0.235) Error: 0.89111 (0.887) Accuracy 0.10888671875
Train: (Epoch 45 of 100) [0667/0860] Time: 0.00583 (4.161) Loss: 0.25918 (0.235) Error: 0.87573 (0.887) Accuracy 0.124267578125
Train: (Epoch 45 of 100) [0668/0860] Time: 0.00621 (4.167) Loss: 0.14686 (0.235) Error: 0.89038 (0.887) Accuracy 0.109619140625
Train: (Epoch 45 of 100) [0669/0860] Time: 0.00612 (4.173) Loss: 0.27873 (0.235) Error: 0.87988 (0.887) Accuracy 0.1201171875
Train: (Epoch 45 of 100) [0670/0860] Time: 0.00649 (4.180) Loss: 0.20269 (0.235) Error: 0.88721 (0.887) Accuracy 0.11279296875
Train: (Epoch 45 of 100) [0671/0860] Time: 0.00661 (4.186) Loss: 0.11891 (0.235) Error: 0.88452 (0.887) Accuracy 0.115478515625
Train: (Epoch 45 of 100) [0672/0860] Time: 0.00604 (4.192) Loss: 0.18655 (0.234) Error: 0.88379 (0.887) Accuracy 0.1162109375
Train: (Epoch 45 of 100) [0673/0860] Time: 0.00604 (4.198) Loss: 0.35968 (0.235) Error: 0.87866 (0.887) Accuracy 0.121337890625
Train: (Epoch 45 of 100) [0674/0860] Time: 0.00626 (4.205) Loss: 0.25359 (0.235) Error: 0.87964 (0.887) Accuracy 0.120361328125
Train: (Epoch 45 of 100) [0675/0860] Time: 0.00626 (4.211) Loss: 0.37408 (0.235) Error: 0.88550 (0.887) Accuracy 0.114501953125
Train: (Epoch 45 of 100) [0676/0860] Time: 0.00641 (4.217) Loss: 0.22567 (0.235) Error: 0.89307 (0.887) Accuracy 0.10693359375
Train: (Epoch 45 of 100) [0677/0860] Time: 0.00633 (4.224) Loss: 0.33593 (0.235) Error: 0.89526 (0.887) Accuracy 0.104736328125
Train: (Epoch 45 of 100) [0678/0860] Time: 0.00619 (4.230) Loss: 0.19604 (0.235) Error: 0.88354 (0.887) Accuracy 0.116455078125
Train: (Epoch 45 of 100) [0679/0860] Time: 0.00603 (4.236) Loss: 0.14126 (0.235) Error: 0.89355 (0.887) Accuracy 0.1064453125
Train: (Epoch 45 of 100) [0680/0860] Time: 0.00578 (4.242) Loss: 0.22659 (0.235) Error: 0.89331 (0.887) Accuracy 0.106689453125
Train: (Epoch 45 of 100) [0681/0860] Time: 0.00628 (4.248) Loss: 0.21508 (0.235) Error: 0.88818 (0.887) Accuracy 0.11181640625
Train: (Epoch 45 of 100) [0682/0860] Time: 0.00615 (4.254) Loss: 0.15964 (0.235) Error: 0.88599 (0.887) Accuracy 0.114013671875
Train: (Epoch 45 of 100) [0683/0860] Time: 0.00603 (4.260) Loss: 0.27280 (0.235) Error: 0.88965 (0.887) Accuracy 0.1103515625
Train: (Epoch 45 of 100) [0684/0860] Time: 0.00593 (4.266) Loss: 0.12067 (0.235) Error: 0.88306 (0.887) Accuracy 0.116943359375
Train: (Epoch 45 of 100) [0685/0860] Time: 0.00636 (4.272) Loss: 0.19499 (0.235) Error: 0.88525 (0.887) Accuracy 0.11474609375
Train: (Epoch 45 of 100) [0686/0860] Time: 0.00652 (4.279) Loss: 0.22604 (0.235) Error: 0.88843 (0.887) Accuracy 0.111572265625
Train: (Epoch 45 of 100) [0687/0860] Time: 0.00633 (4.285) Loss: 0.38691 (0.235) Error: 0.87354 (0.887) Accuracy 0.12646484375
Train: (Epoch 45 of 100) [0688/0860] Time: 0.00614 (4.291) Loss: 0.19034 (0.235) Error: 0.88159 (0.887) Accuracy 0.118408203125
Train: (Epoch 45 of 100) [0689/0860] Time: 0.00618 (4.298) Loss: 0.17172 (0.235) Error: 0.88574 (0.887) Accuracy 0.1142578125