Torch not able to utilize GPU ram properly

[Tyan]

I have a similar problem. I used DistributedDataParallel and python -m torch.distributed.launch --nproc_per_node=8.