I define a class with model initializer. When I wrap one model with that class
it looks fine, but with two model instances, it does not converge.
def __init__(self, model):
self.model = model
def forward(self, x):
model1 = net1()
w_model1 = wrapper(mode1)
model2 = net2()
#when I include this line, w_model1 does not converge.
w_model2 = wrapper(model2)
One thing to keep in mind here is that when you create net2(), you draw random samples. So it has the same effect as changing the random seed for the training of net1. If you net1 model is not very stable, it is possible that it only converge for some seeds.
Otherwise, I can’t really see how this could effect it. If it is not the point above, you would need to provide a sample script that shows the behavior so that we can help you further.