Hello. So I have been doing transfer learning and followed a youtube tutorial but now when I look around to other documents and examples there is this thing very confusing and budging me. I used EfficientNet v1 B0 model and the default model when loaded as

model = timm.create_model(CFG.model_name,pretrained=True) and when i print model the last few layers are as: (conv_head): Conv2d(320, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False)

(bn2): BatchNormAct2d(

1280, eps=0.001, momentum=0.1, affine=True, track_running_stats=True

(drop): Identity()

(act): SiLU(inplace=True)

)

(global_pool): SelectAdaptivePool2d (pool_type=avg, flatten=Flatten(start_dim=1, end_dim=-1))

(classifier): Linear(in_features=1280, out_features=1000, bias=True)

)

Now I have 6 class problem and I want to change last layer features =1000 to 6.

I did this by:

print(model.conv_stem)

model.conv_stem = nn.Conv2d(1, 32, kernel_size=(3, 3), stride=(2, 2), bias=False)

model.state_dict()[‘conv_stem.weight’] = model.state_dict()[‘conv_stem.weight’].sum(dim=1, keepdim=True)

print(model.conv_stem)

#let’s update the pretrained model:

for param in model.parameters():

param.requires_grad=False

#orginally, it was:

#(classifier): Linear(in_features=1280, out_features=1000, bias=True)

model.classifier = nn.Linear(1280, 6)

But this is what I learned but the youtube tutorial says:

model.classifier = nn.Sequential(

nn.Linear(in_features=1792, out_features=625), #1792 is the orginal in_features

nn.ReLU(), #ReLu to be the activation function

nn.Dropout(p=0.3),

nn.Linear(in_features=625, out_features=256),

nn.ReLU(),

nn.Linear(in_features=256, out_features=6)

)

I wonder which is correct and what is feature extractor and what significance sequential thing has? And how one can calculate or decide out_features=625???

Which method is correct whats the pros and cons?