# Two bias vectors in recurrent units?

**URL:** <https://discuss.pytorch.org/t/two-bias-vectors-in-recurrent-units/4612>\
**Category:** Uncategorized\
**Created:** [July 5, 2017, 1:04pm UTC](https://discuss.pytorch.org/t/two-bias-vectors-in-recurrent-units/4612 "2017-07-05T13:04:37Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![magic282](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/magic282/32/831_2.png) [@magic282](https://discuss.pytorch.org/u/magic282)\
**Post date:** [July 5, 2017, 1:04pm UTC](https://discuss.pytorch.org/t/two-bias-vectors-in-recurrent-units/4612/1 "2017-07-05T13:04:37Z")

</div>

Hi,

I just noticed that in recurrent nets, e.g. [http://pytorch.org/docs/master/nn.html#gru](http://pytorch.org/docs/master/nn.html#gru) there are two bias vectors. Is there a particular reason for this?

---

<div class="post-metadata">

**Author:** ![tom](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tom/32/3162_2.png) [@tom](https://discuss.pytorch.org/u/tom)\
**Post date:** [July 5, 2017, 1:13pm UTC](https://discuss.pytorch.org/t/two-bias-vectors-in-recurrent-units/4612/2 "2017-07-05T13:13:55Z")

</div>

Hello,

I think this is to match CUDNN:

> [@Why RNN needs two biases?](https://discuss.pytorch.org/t/why-rnn-needs-two-biases/557/4):
>
> As you pointed out it doesn’t really change the definition of the model, but this is what cuDNN does, so we’ve made our RNNs consistent with this behaviour.

Best regards

Thomas

---

<div class="post-metadata">

**Author:** ![magic282](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/magic282/32/831_2.png) [@magic282](https://discuss.pytorch.org/u/magic282)\
**Post date:** [July 5, 2017, 1:31pm UTC](https://discuss.pytorch.org/t/two-bias-vectors-in-recurrent-units/4612/3 "2017-07-05T13:31:39Z")

</div>

I see. That’s interesting. Don’t understand why CuDNN design it like this.