There are two steps that confuse me in rpn.py:
252 objectness = objectness.detach()
370 proposals = self.box_coder.decode(pred_bbox_deltas.detach(), anchors)
So gradients are detached in both class and box predictions. I don’t understand then, how loss is backpropagated through RPN, because it is computed in RPN and RoI after the detached tensors?