Initialise all model weights the same way each time

[ptrblck]

You could seed to code via torch.manual_seed before creating a new model instance. This would however initialize the global pseudorandom number generator (PRNG) and thus also define the random sequence of every other call into it.
Alternatively, you could initialize the model once, save its state_dict, and reload it afterwards for each new model.