Plot training and validation accuracy and loss of Mask RCNN for instance segmentation

I am using Mask RCNN tutorial at

I want to plot the training and validation accuracy and loss. How can I plot it.

Second problem is that after fine tuning I get a lot of masks. The only valid masks are the top The number of persons in image. All other masks are errors. How can I identify the total valid masks.

You could store the loss and accuracy in a list as scalar values via e.g.:


and use matplotlib.pyplot.plot to visualize the loss and accuracy curves.

You could probably just index the first masks given the number of people. I’m not sure I understand the problem correctly, so could you explain the issue a bit more?

Adding to what @ptrblck said … u can set a threshold value on scores predicted by the model and subsequently filter the corresponding masks … e-g

    outputs = model(image)
    outputs = [{k: for k, v in t.items()} for t in outputs]
    pred_score = list(outputs[0]['scores'].numpy())
    pred_t = [pred_score.index(x) for x in pred_score if x>=threshold][-1]
    masks = (outputs[0]['masks']>0.5).squeeze().numpy()
    masks = masks[:pred_t+1]

i am a little confused …

its okay with the loss, but where is “accuracy” during training in pytorch Mask R-CNN model ??

  # For Training
  images,targets = next(iter(data_loader))
  images = list(image for image in images)
  targets = [{k: v for k, v in t.items()} for t in targets]
  output = model(images,targets)

 {'loss_box_reg': tensor(0.0100, grad_fn=<DivBackward0>),
 'loss_classifier': tensor(0.2265, grad_fn=<NllLossBackward>),
 'loss_objectness': tensor(0.0190, grad_fn=<BinaryCrossEntropyWithLogitsBackward>),
'loss_rpn_box_reg': tensor(0.0068, grad_fn=<DivBackward0>)}

its just a dictionary of losses … they can be plotted but i am more concerned in accuracy plotting during training … any thoughts ?

Depending which accuracy you want you could calculate it using the model output.
How are the targets defined in your use case and which accuracy are you looking for?

Dont we need to have predictions from the model output in order to calculate an accuracy ?? what i was trying to say earlier (and couldnt make it clear) was that for pytorch’s Mask RCNN implementation … we need to have model in eval model in order to generate predictions whcih can be then subsequently used for accuracy calculations … the same cannot be done in model train mode … as said in its documentation here

   The model returns a Dict[Tensor] during training, containing the classification and regression
   losses for both the RPN and the R-CNN, and the mask loss.

and this is realized via eager_outputs methods in generalizedRCNN class as seen here.
To my understanding , in order to get predictions (and subsequent acc) in train mode , the only options seems to be is modification of this eager_outputs method. i have tried it but in vain (as of now).

To answer your question , i am looking for mIoU … which i am already calculting via predictions in eval mode. but since now i want to analyse model overfitting , i need to have a train_mIoU generated from the training dataset. ofcourse i can just change the dataset to training dataset in eval mode to generate train_mIoU but that adds another overhead in computations … i was looking if there is any option to get the same during model training and NOT during evaluations …

P.S thinking out of the box , any other option to analyse model overfitting ? i feel reluctant to go to train/test loss monitoring for this purpose … but open to suggestions .

Hi guys can you please help me to plot the training an d validationn and loss accuracy using example code along with its output. Sorry guys i didnt get the concept of all ur suggestions as i am a beginner to use mask rcnn model for my project.

thanks in advance!!!