# Flatten\_parameters fails with DataParallel?

**URL:** <https://discuss.pytorch.org/t/flatten-parameters-fails-with-dataparallel/20692>\
**Category:** Uncategorized\
**Created:** [July 4, 2018, 9:47am UTC](https://discuss.pytorch.org/t/flatten-parameters-fails-with-dataparallel/20692 "2018-07-04T09:47:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![AlexisW](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/7993a0/32.png) [@AlexisW](https://discuss.pytorch.org/u/AlexisW)\
**Post date:** [July 4, 2018, 9:47am UTC](https://discuss.pytorch.org/t/flatten-parameters-fails-with-dataparallel/20692/1 "2018-07-04T09:47:02Z")

</div>

I have a RNN model. I did something like

```auto
self.lstm.flatten_parameters()

```

with DataParallel on multiple GPUs in order to eliminate a user warning.

```auto
UserWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters().

```

However, I got this error

```auto
RuntimeError: torch/csrc/autograd/variable.cpp:115: get_grad_fn: Assertion `output_nr == 0` failed.

```

Any ideas on this? Is it a bug for pytorch?

---

<div class="post-metadata">

**Author:** ![Shani\_Gamrian](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shani_gamrian/32/1299_2.png) [@Shani\_Gamrian](https://discuss.pytorch.org/u/Shani_Gamrian)\
**Post date:** [July 4, 2018, 1:49pm UTC](https://discuss.pytorch.org/t/flatten-parameters-fails-with-dataparallel/20692/2 "2018-07-04T13:49:15Z")

</div>

What does it say when it fails?

---

<div class="post-metadata">

**Author:** ![AlexisW](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/7993a0/32.png) [@AlexisW](https://discuss.pytorch.org/u/AlexisW)\
**Post date:** [July 4, 2018, 8:01pm UTC](https://discuss.pytorch.org/t/flatten-parameters-fails-with-dataparallel/20692/3 "2018-07-04T20:01:16Z")

</div>

I managed to fix it, but got a new error with `get_grad_fn` (details in the question). Do you have any idea? Thanks!