What happens when you continue before the barrier?

For example:

m = []
for key in adict:
    if key in bdict:
        v = bdict[key]
    else:
        continue
    dist.barrier()
    t = [[None] * len(v) for _ in dist.get_world_size()]
    dist.all_gather_object(t, v)
    m.append(t)

Could the program be stuck because a process that does not contain the key has already entered the next loop and the barrier can never wait for this process to enter?


I’m trying a handler similar to the above example:

key = ['a', 'b', 'c']
for k in key:
    if k in new_dict:
        # gather new_dict[k] from processes
    dist.barrier()

There is a possibility that there is no certain key in the new_dict. But this program gets stuck.

Yes, it can. All peer processes need to call into the same set of collectives in the same order.