Hi All, I am trying to freeze a lot of layers except few layers (in below came fc_cat and fc_kin_lin are two linear layers) while keeping the dropout layers on and Batch norm layer turned off. Is this a right way to do it? I know there are different posts with each of them but not together. de…

Replying to my own question I realized that I could use name_paratmer instead of state_dict parameter_dict = dict(model.named_parameters()) for name, param in parameter_dict.items(): if name.find('fc_cat')>0 or name.find('fc_kin_lin')>0: print('\n\n', name, 'layer parameters will be tr…

hi @Gurkirt , I am also stuck in a somewhat similar issue. Can you please suggest me if you have solved yours? Here is my issue: [image] Use global statistics of BatchNorm in training mode I am working on a edge detection code and my batch_size=1 (due to GPU RAM’s limit…

Hey @deepak242424 You can use model.apply(function_name) here for batch norm layers `def set_bn_eval(m): classname = m.__class__.__name__ if classname.find('BatchNorm') != -1: m.eval()` then call this function by .apply before starting the training. model.train() model.apply(set_bn_eval) …

Finetune subset of parameters with BatchNorm turned off

deepak242424 (Deepak Mittal) October 31, 2018, 9:06am 3

hi @Gurkirt,

I am also stuck in a somewhat similar issue. Can you please suggest me if you have solved yours?
Here is my issue: