Overfitting a segmenation network. Validation performs better than train

1

Hi!

I am trying to train various types of segmentation network. Currently I am trying to overfit a dataset, consisting of 3 tensors. Two of them are torch.ones of dim 3x192x192 and the last one is torch.zero of dim 3x192x192. For 200 epochs i am getting the following:

current learning rate : 0.0001
Phase: train. Metrics for current epoch: 199:
current epoch 199 – recall : 0.9502495659722897
current epoch 199 – precision : 0.5000000000043909
current epoch 199 – accuracy : 0.8979017469620487
current epoch 199 – dice_score : 0.47386653059730577
current epoch 199 – loss : 0.6017277367443301
current learning rate : 0.0001
Phase: validation. Metrics for current epoch: 199:
current epoch 199 – recall : 0.9998101128472228
current epoch 199 – precision : 1.0
current epoch 199 – accuracy : 0.9999050564236114
current epoch 199 – dice_score : 0.9999050486964542
current epoch 199 – loss : 0.05479495529160455

As you can see, train perform way poorer than validation. I am using the same tensor for validation and train.

2

A long shot, but do you have Dropout layers in your architecture? If so, perhaps those are messing up your training loss. Dropout gets turned off in model eval mode (for validation) so that’s a candidate for something that could cause a model to do better in validation than in training.

1 Like