No Bias in the pretrianed state dictionary of ResNet18

vision
1

I would like to load my model which is modified with the pre-trained weights of a ResNet18. I noticed that there is no bias in the state_dict of this model and bias=False in the conv2d layer. My questions are:

1 - is it possible to access to a version of these pre-trained models in PyTorch that have bias values too?
2 - Why pre-trained bias values are not available and what is the effect of having informed weight values but randomly initialized bias vectors?

I appreciate your guidance and opinions.

2

In general, ResNet’s conv layers are followed by batch norm layers which perform mean subtraction. This mean subtraction step will effectively remove the influence of bias. Hence, bias is not used in conv layers (even if bias is used, their gradients would be almost 0).

1 Like