Average Precision and Loss decrease


I am using running Maskrcnn and using AP as a metric. After a certain number of epochs, the loss continues decreasing. However, the APcollapses. What can I do to solve this issue?
Please advise.




What is the loss function you are using? Could post a snippet of your code?

Hi @Preetham_R_Patlolla
I am using engine.py from the references/detection. So this is the sum of all losses for segmentation and detection. I am also using Adam optimizer.