# LSTM/GRU gate weights

**URL:** <https://discuss.pytorch.org/t/lstm-gru-gate-weights/2807>\
**Category:** Uncategorized\
**Created:** [May 9, 2017, 10:08pm UTC](https://discuss.pytorch.org/t/lstm-gru-gate-weights/2807 "2017-05-09T22:08:57Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![aron-bordin](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aron-bordin/32/505_2.png) [@aron-bordin](https://discuss.pytorch.org/u/aron-bordin)\
**Post date:** [May 9, 2017, 10:08pm UTC](https://discuss.pytorch.org/t/lstm-gru-gate-weights/2807/1 "2017-05-09T22:08:57Z")

</div>

Hi 🙂

I would like to have a custom weight initialization to each gate of my rnn (GRU and LSTM). How can I get the weights of a specific gate in the GRU/LSTM implementation ?

---

<div class="post-metadata">

**Author:** ![ruotianluo](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ruotianluo/32/383_2.png) [@ruotianluo](https://discuss.pytorch.org/u/ruotianluo)\
**Post date:** [May 10, 2017, 1:27pm UTC](https://discuss.pytorch.org/t/lstm-gru-gate-weights/2807/2 "2017-05-10T13:27:25Z")

</div>

```
#LSTM
net = nn.LSTM(100, 100) #Assume only one layer
w_ii, w_if, w_ic, w_io = net.weight_ih_l0.chunk(4, 0)
w_hi, w_hf, w_hc, w_ho = net.weight_hh_l0.chunk(4, 0)

#GRU
net = nn.GRU(100,100)
w_ir, w_ii, w_in = net.weight_ih_l0.chunk(3, 0)
w_hr, w_hi, w_hn = net.weight_ih_l0.chunk(3, 0)

```

reference:

> <https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/rnn.py>

---

<div class="post-metadata">

**Author:** ![Dheeraj\_M\_Pai](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/d07c76/32.png) [@Dheeraj\_M\_Pai](https://discuss.pytorch.org/u/Dheeraj_M_Pai)\
**Post date:** [July 3, 2018, 4:40pm UTC](https://discuss.pytorch.org/t/lstm-gru-gate-weights/2807/3 "2018-07-03T16:40:18Z")

</div>

How do I get the biases?

---

<div class="post-metadata">

**Author:** ![purvanshi\_mehta](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/purvanshi_mehta/32/5724_2.png) [@purvanshi\_mehta](https://discuss.pytorch.org/u/purvanshi_mehta)\
**Post date:** [July 19, 2018, 4:18pm UTC](https://discuss.pytorch.org/t/lstm-gru-gate-weights/2807/4 "2018-07-19T16:18:59Z")

</div>

You can get weights of all layers by

```auto
rnn = nn.LSTM(100, 100,20)
for name in rnn.named_parameters():
    if 'weight' in name[0]:
        weight_list.append(name[1])

```

---

<div class="post-metadata">

**Author:** ![Kris](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/82dd89/32.png) [@Kris](https://discuss.pytorch.org/u/Kris)\
**Post date:** [October 3, 2018, 12:23pm UTC](https://discuss.pytorch.org/t/lstm-gru-gate-weights/2807/6 "2018-10-03T12:23:03Z")

</div>

Great answer, thanks!  
But I think that for the GRU, the second `weight_ih_l0` should be changed to `weight_hh_l0`

---

<div class="post-metadata">

**Author:** ![miranda](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/miranda/32/21836_2.png) [@miranda](https://discuss.pytorch.org/u/miranda)\
**Post date:** [March 18, 2020, 5:50pm UTC](https://discuss.pytorch.org/t/lstm-gru-gate-weights/2807/7 "2020-03-18T17:50:09Z")

</div>

@ [Dheeraj\_M\_Pai](https://discuss.pytorch.org/u/Dheeraj_M_Pai): You can get biases by: `net.bias_hh_l0`, and `net.bias_ih_l0`. 0 in `l0` stands for layer number. If you have more than one lstm layer then change 0 to 1, 2, or the number of whichever layer you want to manipulate.