I am trying to use transfer learning for image classification.
Can anyone suggest best practices,
I have seen multiple tutorial where few people adds single fc layers after removing the initial fc layer
Also, it seems like a rabbit hole where we can freeze-unfreeze the different number of sub-modules. It seems not very efficient to me.
Can you guys share your best practices, thumb rule etc. (I know it is a very comprehensive question but I am thinking a lot and really not able to converge at a precise mental model on how to approach it.)
I don’t think there is a simple answer to this question.
I think that’s unfortunately still often the case and you will read in a lot of papers that the training hyperparameters were “tuned” or “heuristically determined”.
You could have a look at e.g. FastAI which explains the best practices for some use cases.