# GPU memory increases as output sequence increases

**URL:** <https://discuss.pytorch.org/t/gpu-memory-increases-as-output-sequence-increases/8791>\
**Category:** Uncategorized\
**Created:** [October 18, 2017, 7:31am UTC](https://discuss.pytorch.org/t/gpu-memory-increases-as-output-sequence-increases/8791 "2017-10-18T07:31:39Z")\
**Posts on this page:** 1\
**Showing post:** 11

<div class="post-metadata">

**Author:** ![Ben](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/df788c/32.png) [@Ben](https://discuss.pytorch.org/u/Ben)\
**Post date:** [October 26, 2017, 10:26am UTC](https://discuss.pytorch.org/t/gpu-memory-increases-as-output-sequence-increases/8791/11 "2017-10-26T10:26:50Z")

</div>

I read this post. My issue is similar to the second problem in it.

> [@Understanding graphs and state](https://discuss.pytorch.org/t/understanding-graphs-and-state/224/2):
>
> Yes. We don’t guarantee that the error will be raised, but if you want to be sure that you can backprop multiple times you need to specify retain\_variables=True. It won’t raise an error only for very simple ops like the ones you have here (e.g. grad\_input of add is just grad\_output, so there’s no need for any buffers, and that’s why it also doesn’t check if they were freed). Not sure if we should add these checks or not. It probably doesn’t matter, as it will raise a clear error, and otherwise…

I actually feedback the output as part of the input to feed into the model for tens of times, and then calculate the loss. If the model graph and its variables are generated in every loop, that will take much memory. I’m not sure if this will lengthen the backward path and make the backprop harder.

In tensorflow, I can use while\_loop to let a model to run for several times and then calculate the loss.

---

_[View the full topic](https://discuss.pytorch.org/t/gpu-memory-increases-as-output-sequence-increases/8791)._