How to train model to avoid forgetting of previous knowledge?

autograd
1

Hi there,

I have a model with trained weights W and I want to train that model on different task without forgetting previous knowledge. I have another matrix V of the same size as W, and

# A.requires_grad=True
# B.requires_grad=False
V = torch.matmul(A, B) # then V.requires_grad=True

New Weights W1 = W + V so W1 has automatically requires_grad=True and I don’t want to update W1, W, V, and B I only want to update A, How I can do that?

Thanks!

2

Sorry, Solve it. It’s already there. I got confused previously.

Thanks!