Hey guys,
I’m working with a simple model which has to train 1 value only, named self.my_weights
in my below code. The only new thing in my experiment is that I have to round that value when forwarding the input.
In my observation, that value doesn’t change anything when training. Assume the loss function is working. Does my “rounding values” action make that tensor un-trainable? Do I have any wrong implementations? Pls help!
This is an example of my network:
class MyModel(nn.Module):
def __init__(self):
super(MyModel, self).__init__()
self.backbone = MobileNetv3(pretrain=True) #This self.backbone's weights are frozen when training
self.backbone.classifier = nn.Sequential()
self.my_weights = nn.Parameter(torch.ones(1, requires_grad=True))
def forward(self, input):
feature = self.backbone(input)
repeat_weights = torch.round(self.features_weights).to(torch.int64)
final_features = feature.repeat(1, repeat_weights[0]))
final_features = F.normalize(final_features, dim=1)
return final_features