How to create a branch in ResNet with ImageNet pretrained weights?

vision
1

image
image645×607 13.8 KB

I want to create a ResNet-18 model with two branches as shown in the above figure, where I replicate the original Layer 4 to create the branches. However, I don’t know how to initialize the weights of these two layers (Layer 4_1 and Layer 4_2) with ImageNet pretrained weights. Could you please help me with this.

2

You could manually load all desired parameters by directly accessing the modules.
E.g. something like this would work to load the parameters into the two custom layers:

model = models.resnet18()
sd = model.state_dict()

layer4_1 = nn.Conv2d(256, 512, 3, 2, 1, bias=False)

# access custom layers and load parameters manually
with torch.no_grad():    
    layer4_1.weight.copy_(sd["layer4.0.conv1.weight"])

Note that you won’t be able to load the entire state_dict since you’ve changed the model architecture. You could use strict=False, but I would highly recommend checking the mismatches or missing keys explicitly.