Initialize Model vs Transfer Learning

I have the following code:

for i in range(num_models):

    model = ConvNet()
    model.cuda()
    model = nn.DataParallel(model, device_ids=[0])

    ...

I suspect that these lines don’t actually reinitialize the model with random weights. I believe this because I observed better results for models which are trained in later iterations of the for-loop than if the same models were trained individually without any for-loop, leading me to think that some sort of transfer learning is taking place.

Is this a possibility? How could I reinitialize the weights of the model to be random with every iteration of the for-loop?

The model is randomly initialized in each loop. You could run a quick test and compare (some) parameters of the new model to the last trained one (use .clone() to get the real value instead of a reference).