LSTM/GRU gate weights

(Aron Bordin) #1

Hi :slight_smile:

I would like to have a custom weight initialization to each gate of my rnn (GRU and LSTM). How can I get the weights of a specific gate in the GRU/LSTM implementation ?

(Ruotian(RT) Luo) #2
net = nn.LSTM(100, 100) #Assume only one layer
w_ii, w_if, w_ic, w_io = net.weight_ih_l0.chunk(4, 0)
w_hi, w_hf, w_hc, w_ho = net.weight_hh_l0.chunk(4, 0)

net = nn.GRU(100,100)
w_ir, w_ii, w_in = net.weight_ih_l0.chunk(3, 0)
w_hr, w_hi, w_hn = net.weight_ih_l0.chunk(3, 0)


(Dheeraj M Pai) #3

How do I get the biases?

(Purvanshi Mehta) #4

You can get weights of all layers by

rnn = nn.LSTM(100, 100,20)
for name in rnn.named_parameters():
    if 'weight' in name[0]:


Great answer, thanks!
But I think that for the GRU, the second weight_ih_l0 should be changed to weight_hh_l0