# DataParallel imbalanced memory usage

**URL:** <https://discuss.pytorch.org/t/dataparallel-imbalanced-memory-usage/22551>\
**Category:** Uncategorized\
**Created:** [August 6, 2018, 10:29pm UTC](https://discuss.pytorch.org/t/dataparallel-imbalanced-memory-usage/22551 "2018-08-06T22:29:49Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![soulless](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/soulless/32/40008_2.png) [@soulless](https://discuss.pytorch.org/u/soulless)\
**Post date:** [August 8, 2018, 4:47am UTC](https://discuss.pytorch.org/t/dataparallel-imbalanced-memory-usage/22551/2 "2018-08-08T04:47:24Z")

</div>

I am having a similar issue. Could this be because the loss calculation is not done in the forward function?

> [@Imbalanced GPU memory usage training LSTM](https://discuss.pytorch.org/t/imbalanced-gpu-memory-usage-training-lstm/15049/2):
>
> I found the reason, it is because we collect the output back to one gpu and calculate loss there. If move loss calculation into model.forward(), the problem is resolved.

---

_[View the full topic](https://discuss.pytorch.org/t/dataparallel-imbalanced-memory-usage/22551)._