Tweaks from transfer learning model

currently I am using a pre-trainned model from torchvision to after fine-tunning it to my dataset.
My goal is to tweak the model and see what the differences causes in the performance.
But if I use pre-trainned model of squeezeNet and after I add batch normalization it gets me an error

KeyError: ‘unexpected key “features.3.squeeze.weight” in state_dict’

How can I bypass that? Currently I just edit the model file and add manually the batch normalization. I can’t add it trough features.add_module since I want to keep the weights. How could I proceed?
Thank you for your time.