From documentation http://pytorch.org/docs/master/nn.html#torch.nn.LSTM, LSTM module carries variable weight_ih_l[k]
. How can I access the variable? I tried things like rnn.weight_ih_l[0]
, in which rnn
is the name of my LSTM module, and it throws error:
Traceback (most recent call last):
File "main.py", line 194, in <module>
train()
File "main.py", line 165, in train
loss += add_dimension_glasso(model.rnn.weight_ih_l[0], 0)
File "/Users/evan/anaconda2/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py", line 262, in __getattr__
type(self).__name__, name))
AttributeError: 'LSTM' object has no attribute 'weight_ih_l'
It is easy to reproduce the error. I use skeleton from https://github.com/pytorch/examples/tree/master/word_language_model, and in main.py
, I add the following function:
def add_dimension_glasso(var, axis=0):
return var.pow(2).sum(axis=axis).add(1e-8).pow(1/2.).sum()
and modified train()
a little bit, with only 1 line added:
loss = criterion(output.view(-1, ntokens), targets)
loss += add_dimension_glasso(model.rnn.weight_ih_l[0], 0) # added code!
loss.backward()
Can anyone help me with this? I simply want to get weight_ih_l. Thanks!
PS:
$ python -c "import torch; print(torch.__version__)"
0.2.0_4
Edit: solved.
I print model.rnn.state_dict()
to find out I should use weight_ih_l0
and weight_ih_l1
(given 2 layers lstm). I feel the way documentation shows is somewhat misleading.