Seeds for Dataloader

1

I am looking at this page to understand how I can aspire to reproducibility:
https://pytorch.org/docs/stable/notes/randomness.html

Specifically, this code:

def seed_worker(worker_id):
    worker_seed = torch.initial_seed() % 2**32
    numpy.random.seed(worker_seed)
    random.seed(worker_seed)

g = torch.Generator()
g.manual_seed(0)

DataLoader(
    train_dataset,
    batch_size=batch_size,
    num_workers=num_workers,
    worker_init_fn=seed_worker,
    generator=g,
)

I cannot understand this line:

g.manual_seed(0)

Questions:

  • has 0 a special meaning?
  • what if I want to use the seed 1?

Suppose I have already set the seed to the value 1 as indicated at the beginning of the same page, so that torch.initial_seed() == 1. Should I keep 0 in that line?

2

Hello! The line g.manual_seed(0) is setting the seed of the random number generator g to 0.
Just to let you know, I was looking at the tutorial and 0 does not have any special meaning but is just used throughout the tutorial, so it will make sense to stick with 0. It is simply an example seed value used in the code.

Should I keep 0 in that line if torch.initial_seed() == 1?

No, you should set the seed in g.manual_seed() to match the seed you want to use throughout your code for reproducibility. If you have already set the seed to 1 using torch.manual_seed(1) at the beginning, you should use the same seed value consistently.