It seems that wrapping my classes in DataParallel(model)
, I see there’s an imbalance between the allocated memory on each GPU. Is there a way to balance them more? It seems this imbalance is making it more likely to cause OOM errors
This is expected as it’s a known drawback of nn.DataParallel
which is one of the reasons why we recommend using DistributedDataParallel
instead.
1 Like