Loading old FastAI XResNet50 pretrained weights into newer XResNet implementation

vision
1

Loading old FastAI XResNet50 pretrained weights into newer XResNet implementation

Hi,

I am using FastAI with xresnet50 as the encoder of a DynamicUnet, and I am having trouble correctly loading the FastAI pretrained XResNet50 weights.

The pretrained checkpoint is:

xrn50_940.pth

from the FastAI model zoo.

The checkpoint appears to use an older XResNet naming structure, for example:

4.0.convs.0.0.weight
4.0.idconv.0.weight

while the XResNet50 model in my FastAI installation uses keys such as:

4.0.convpath.0.0.weight
4.0.idpath.0.0.weight

Therefore, loading the checkpoint directly with load_state_dict(..., strict=False) results in many missing and unexpected keys.

I currently solve this by manually remapping the checkpoint keys:

new_key = old_key.replace(".convs.", ".convpath.")

if new_key.startswith("4.0.idconv."):
    new_key = new_key.replace(
        "4.0.idconv.",
        "4.0.idpath.0."
    )

elif new_key.startswith("5.0.idconv."):
    new_key = new_key.replace(
        "5.0.idconv.",
        "5.0.idpath.1."
    )

elif new_key.startswith("6.0.idconv."):
    new_key = new_key.replace(
        "6.0.idconv.",
        "6.0.idpath.1."
    )

elif new_key.startswith("7.0.idconv."):
    new_key = new_key.replace(
        "7.0.idconv.",
        "7.0.idpath.1."
    )

I only load a tensor when both its key and shape match the current model.

With this approach I get:

Loaded pretrained tensors: 329
Missing keys: ['0.0.weight']
Unexpected keys: []

The classifier tensors from the original checkpoint are intentionally not used because I only need the XResNet50 encoder for the U-Net.

I also use multispectral input rather than RGB, so the first convolution eventually needs more than 3 input channels.

My questions are:

  1. Is manually remapping convs -> convpath and idconv -> idpath the correct way to use the old xrn50_940.pth weights with the newer FastAI XResNet implementation?
  2. Are the mappings 4.0.idconv -> 4.0.idpath.0 and 5/6/7.0.idconv -> 5/6/7.0.idpath.1 correct?
  3. Is there an official/recommended FastAI method for loading this older XResNet50 checkpoint instead?
  4. For a multispectral U-Net encoder, what is the recommended way to adapt the pretrained 3-channel first convolution to N input channels while preserving as much of the pretrained information as possible?

I would especially like to verify that the encoder is genuinely initialized with the intended pretrained XResNet50 weights before training the DynamicUnet.

Thanks!