Different init for training ensembles

vision
1

Currently am fixing the all seeds by doing the following,

random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
np.random.seed(seed)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

I want to train multiple models of the same architecture type but with different initialization and ensemble them later.

I usually change the seed to get a different init but since am fixing the seed for everything, how do I still get a different init?

2

Setting the seed at the beginning of the script would make the pseudorandom number generator output deterministic “random” values. Creating multiple models in the same script would thus also create different parameters, since the sequence of the random number generation is defined by the seed, but the values won’t be the same.

1 Like