How can l load my best model as a feature extractor/evaluator?

You could iterate all modules and register hooks with them as seen here.
I’m not sure, if there is another cleaner way of checking for “layers” besides the used if condition.