Add custom Layer in the middle of torchvision model

You could replace the layer with an nn.Sequential block containing the original layer as well as your new layer as described here.