Change parameters for Swin-transformer

vision
1

I’m trying to change the Swin transformer parameter DROP_PATH_RATE to fine-tune by doing:

model = torchvision.models.swin_b(weights=torchvision.models.Swin_B_Weights.IMAGENET1K_V1, stochastic_depth_prob=0.2)

but I’m finding that it seems to be fixed in swin_transfomer.py.

return _swin_transformer(
        patch_size=[4, 4],
        embed_dim=128,
        depths=[2, 2, 18, 2],
        num_heads=[4, 8, 16, 32],
        window_size=[7, 7],
        stochastic_depth_prob=0.5,
        weights=weights,
        progress=progress,
        **kwargs,
    )

Are there any ways to define the DROP_PATH_RATE parameter when initializing the Swin transformer?

2

I answered myself.
For Swin Transformer base with pre-trained ImageNet-1K:

weights = torchvision.models.Swin_B_Weights.IMAGENET1K_V1
model = torchvision.models.SwinTransformer(
        patch_size=[4, 4],
        embed_dim=128,
        depths=[2, 2, 18, 2],
        num_heads=[4, 8, 16, 32],
        window_size=[7, 7],
        stochastic_depth_prob=0.2
)
model.load_state_dict(weights.get_state_dict(progress=True, check_hash=True))