How do we then make the weights learnable? I was looking at this discussion. Won’t those weights also be minimized? or is it because they are using nn.parameter(), it will be ok? @ptrblck
Basically: I understand that adding weights as learnable is a bad idea, because the model is going to find that making the weights zero is the best optimization. However, what i dont understand is what they are doing in that discussion pasted above.