How apply loss on weights from two CNNs?

I am working on a project in which I have a loss like (w - wc)^2. w is the weight of the CNN that I’m training on, wc is the weight from the pre-trained network, is it possible for me to apply a L2 loss on these two weights in my loss? Will pytorch update w automatically? Thanks.

It is possible. Calculate it as your usual loss. Suppose the CNNs are exactly the same except weight values. You can do something like:

weight_loss = sum((w1 - w2).pow_(2).mean() for w1, w2 in zip(net1.parameters(), net2.parameters()))

Then let your optimizer to its trick.