LSTM/GRU gate weights


(Aron Bordin) #1

Hi :slight_smile:

I would like to have a custom weight initialization to each gate of my rnn (GRU and LSTM). How can I get the weights of a specific gate in the GRU/LSTM implementation ?


(Ruotian(RT) Luo) #2
#LSTM
net = nn.LSTM(100, 100) #Assume only one layer
w_ii, w_if, w_ic, w_io = net.weight_ih_l0.chunk(4, 0)
w_hi, w_hf, w_hc, w_ho = net.weight_hh_l0.chunk(4, 0)

#GRU
net = nn.GRU(100,100)
w_ir, w_ii, w_in = net.weight_ih_l0.chunk(3, 0)
w_hr, w_hi, w_hn = net.weight_ih_l0.chunk(3, 0)

reference:


(Dheeraj M Pai) #3

How do I get the biases?


(Purvanshi Mehta) #4

You can get weights of all layers by

rnn = nn.LSTM(100, 100,20)
for name in rnn.named_parameters():
    if 'weight' in name[0]:
        weight_list.append(name[1])


(Purvanshi Mehta) #5
rnn = nn.LSTM(100, 100,20)
for name in rnn.named_parameters():
    if 'bias' in name[0]:
        bias_list.append(name[1]