Why these two models(fasterrcnn_resnet50_fpn ) perfomance differently?

I built a fasterrcnn_resnet50_fpn by myself, originally I want to change anchor size, while after I keep same anchor size as official tutoril model


(https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html)

I found there are some different outcome.

The model I built:
######################################################
from torchvision.models.detection.faster_rcnn import FasterRCNN
from torchvision.models.detection.rpn import AnchorGenerator
from torchvision.models.detection.backbone_utils import resnet_fpn_backbone
import torchvision

backbone = resnet_fpn_backbone(‘resnet50’, pretrained=True)
backbone.out_channels = 256
anchor_sizes = ((16,), (24,), (32,), (40,), (48,))
aspect_ratios = ((0.5, 1.0, 2.0),) * len(anchor_sizes)
anchor_generator = AnchorGenerator(anchor_sizes, aspect_ratios)
model_define = FasterRCNN(backbone, num_classes=2,rpn_anchor_generator=anchor_generator)
#####################################################################
In model architecture, I can’t find any difference between these two model, while the outcoms are:

(from my model)

Epoch: [19] [0/5] eta: 0:00:19 lr: 0.000000 loss: 0.7908 (0.7908) loss_classifier: 0.0189 (0.0189) loss_box_reg: 0.0000 (0.0000) loss_objectness: 0.5276 (0.5276) loss_rpn_box_reg: 0.2442 (0.2442) time: 3.9512 data: 3.1582 max mem: 6999
Epoch: [19] [4/5] eta: 0:00:01 lr: 0.000000 loss: 0.7908 (0.8584) loss_classifier: 0.0268 (0.0330) loss_box_reg: 0.0001 (0.0001) loss_objectness: 0.5352 (0.5403) loss_rpn_box_reg: 0.2442 (0.2850) time: 1.4091 data: 0.6444 max mem: 6999

IoU metric: bbox
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

=================================================================
(from official tutorial model)

Epoch: [19] [0/5] eta: 0:00:08 lr: 0.000000 loss: 0.5121 (0.5121) loss_classifier: 0.0884 (0.0884) loss_box_reg: 0.0598 (0.0598) loss_objectness: 0.1395 (0.1395) loss_rpn_box_reg: 0.2244 (0.2244) time: 1.7792 data: 1.3481 max mem: 1985
Epoch: [19] [4/5] eta: 0:00:00 lr: 0.000000 loss: 0.3280 (0.3655) loss_classifier: 0.0746 (0.0719) loss_box_reg: 0.0558 (0.0446) loss_objectness: 0.0961 (0.1129) loss_rpn_box_reg: 0.1067 (0.1361) time: 0.8701 data: 0.5613 max mem: 1985

IoU metric: bbox
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.046
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.152
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.010
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.049
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.090
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.077
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.219
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.296
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.336
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.122
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

I try many times, while the outcomes from my model are always same like this:

============================================================
IoU metric: bbox
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] =0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

Plz help me, I’m so confused :joy: :crazy_face: :thinking:

Could you explain a bit what you’ve changed in your custom model?

PS: You can post code snippets by wrapping them into three backticks ```, which would make debugging a lot easier. :wink: