I have the following code:
for i in range(num_models): model = ConvNet() model.cuda() model = nn.DataParallel(model, device_ids=) ...
I suspect that these lines don’t actually reinitialize the model with random weights. I believe this because I observed better results for models which are trained in later iterations of the for-loop than if the same models were trained individually without any for-loop, leading me to think that some sort of transfer learning is taking place.
Is this a possibility? How could I reinitialize the weights of the model to be random with every iteration of the for-loop?