I’m confused about how to set the seed in torch DDP. I know that the models must be initialized to the same parameter values across processes, so the seed must be the same.
Looking at the code from the DeiT repo I see the line
seed = args.seed + utils.get_rank().
Doesn’t this mean we have a different seed for each process? So wouldn’t the models in each process be initialized differently?