@mrshenli The dist.barrier() doesn’t work while running validation only on rank 0, as mentioned in this thread.
dist.barrier()