Faster RCNN with inceptionv3 backbone very slow


I want to train a faster-rcnn model with an InceptionV3 backbone. I have managed to produce code that works, the problem is however that it trains very slow in comparison to other backbones. My code looks as follows:

def initiate_inceptionv3(num_classes):
        inception = torchvision.models.inception_v3(pretrained=True, aux_logits=False)
        modules = list(inception.children())[:-1]
        backbone = nn.Sequential(*modules)

        for layer in backbone:
            for p in layer.parameters():
                p.requires_grad = False

        backbone.out_channels = 2048

        anchor_generator = AnchorGenerator(sizes=((32, 64, 128, 256, 512),),
                                           aspect_ratios=((0.5, 1.0, 2.0),))

        model = FasterRCNN(backbone, rpn_anchor_generator=anchor_generator,
                           box_predictor=FastRCNNPredictor(1024, num_classes))

        return model

Any suggestion as to why it runs so slow and how to improve? Thanks in advance