Does FSDP support uneven inputs like DDP using Join Hook

distributed
1

Does FSDP support uneven inputs like DDP using Join Hook

2

It does not support the join hook.

3

We have found that most people solve the uneven-batch-size-per-rank issue at the dataloader level now.

2 Likes
4

What’s the most common method of resolving it at the dataloader level?

5

The dataloader ensures every rank gets the same local batch size.

2 Likes
6

Thanks for your reply.