I don't understand WHY don't just use nn.Identity()?

1

image

Here in pytorch-image-models/timm/layers/patch_embed.py at main · huggingface/pytorch-image-models (github.com) people in timm set norm_layer = None
Than write:

 self.norm = norm_layer(hidden_features) if norm_layer is not None else nn.Identity()

Question is why don’t just set norm_layer to nn.Identity()?

2

Hi @nnurseultan07,

This seems to be a general fallback if a particular norm layer isn’t specified, so if you pass norm_layer = nn.Identity(), you’d get the same behavior as setting it to norm_layer = None. I’d assume this is just a way of having a ‘default’ norm_layer (which does just the identity) in the case no norm_layer is selected, but you’d get the same behavior if you pass norm_layer = nn.Identity().