I’m adapting a 10-class classification model to an 11-class classification model where I add one extra class. As well as keeping the prior layer weights, I want to make use of as much of the final classifier weights as possible, so I do something like this:
def make_backbone(self, load=''):
self.backbone = SFNetV1()
if len(load):
self.backbone.load(load)
# replace fully connected layer
if len(load):
prior_weight = self.backbone.fc1.weight
prior_bias = self.backbone.fc1.bias
prior_in_features = self.backbone.fc1.in_features
prior_out_features = self.backbone.fc1.out_features
# add "empty" token
self.backbone.fc1 = nn.Linear(prior_in_features, prior_out_features+1)
print(self.backbone.fc1.weight.is_leaf)
print(prior_weight.is_leaf)
# reuse whatever weights and biases we still can
if len(load):
self.backbone.fc1.weight[:prior_out_features, :] = prior_weight
self.backbone.fc1.bias[:prior_out_features] = prior_bias
print(self.backbone.fc1.weight.is_leaf)
This outputs
True
True
False
So my problem is that last False
. So how do I “reset” self.backbone.fc1.weight
to be a leaf node (also bias
)?
Bonus side question: Is there a better way to do what I’m trying to do?