There is a pytorch implementation of the model described in the paper “Hierarchical Attention Networks for Document Classification” paper. in gihub. When I was using my own data, there was an error：In the forward propagation process, the feature changes from two-dimensional [btach_size and future_dim] to one-dimensional [future_dim].
43
fature: torch.Size([128, 200])
tensor([[-0.0094, -0.0273, -0.0221, ..., 0.0299, 0.0019, -0.1354],
[-0.0314, -0.0314, -0.0286, ..., 0.0330, 0.0025, -0.0863],
[-0.0149, -0.0274, -0.0402, ..., 0.0417, 0.0114, -0.0832],
...,
[-0.0280, -0.0274, -0.0387, ..., 0.0508, -0.0074, -0.1086],
[-0.0135, -0.0102, -0.0637, ..., 0.0753, 0.0128, -0.0639],
[-0.0204, -0.0274, -0.0380, ..., 0.0388, 0.0185, -0.0865]])
44
fature: torch.Size([200])
tensor([ 0.0391, 0.0423, -0.0006, 0.0140, -0.0563, -0.0199, 0.0343, 0.0095,
0.0262, 0.0202, -0.0553, 0.0087, -0.0835, -0.0059, -0.0274, 0.0213,
0.0568, -0.0260, 0.0170, 0.0073, 0.0744, 0.0198, 0.0064, 0.0348,
-0.0576, 0.0424, -0.0401, -0.0294, 0.0116, 0.0395, 0.0107, -0.0550,
-0.1116, 0.0902, -0.0510, 0.0082, -0.0073, 0.0013, 0.0503, -0.0314,
0.0340, -0.0393, 0.0026, 0.0085, 0.0219, 0.0207, 0.0008, -0.0266,
0.0642, 0.0353, 0.0158, -0.0266, -0.0135, -0.0798, 0.0170, 0.0014,
0.0501, -0.0407, -0.1198, -0.0407, 0.0132, -0.0408, -0.0646, 0.0879,
-0.0117, -0.0062, 0.0437, 0.0289, -0.0196, -0.0293, -0.0125, 0.0805,
0.0368, -0.0111, -0.0031, -0.0010, 0.0388, -0.0483, -0.0001, 0.0167,
-0.0459, -0.0234, -0.0038, 0.0443, 0.0501, -0.0359, -0.0395, -0.0486,
0.0451, -0.0089, 0.0270, -0.0063, -0.0079, -0.0057, 0.0067, -0.0185,
0.0217, 0.0746, -0.0117, 0.0175, -0.0089, -0.0106, -0.0345, -0.0077,
-0.0411, -0.0724, 0.0220, -0.0212, -0.0246, 0.0863, 0.0113, -0.0067,
0.0016, -0.0506, 0.0203, 0.0210, -0.0541, -0.0059, -0.0213, 0.0214,
0.0144, -0.0114, 0.0491, -0.0107, -0.0105, 0.0096, -0.0364, -0.0067,
-0.0130, 0.0528, -0.0035, -0.0092, 0.0198, 0.0031, -0.0109, 0.0033,
0.0193, -0.0492, 0.0038, 0.0780, -0.0177, 0.0428, 0.0174, -0.0387,
0.0426, -0.0266, -0.0248, 0.0047, -0.0175, -0.0035, 0.0937, 0.0171,
-0.0195, 0.0106, 0.0133, 0.0472, 0.0144, -0.0754, -0.0273, -0.0104,
-0.0112, 0.0168, 0.0012, 0.0329, 0.0295, 0.0017, -0.0366, 0.0655,
0.0358, 0.0519, 0.0016, -0.0390, -0.0407, 0.0162, -0.0045, 0.0121,
0.0123, 0.0083, 0.0304, 0.0358, 0.0617, -0.0152, -0.0165, -0.0002,
-0.0001, -0.0143, -0.0056, 0.0038, 0.0150, 0.0724, 0.0733, 0.0152,
0.0667, 0.0588, -0.0420, 0.0514, -0.0170, 0.0184, 0.0426, -0.0109])
Then I got an error message.
Traceback (most recent call last):
File "D:/nlp/code/Hierarchical_Attention_Networks_PyTorch/train.py", line 157, in <module>
train(opt)
File "D:/nlp/code/Hierarchical_Attention_Networks_PyTorch/train.py", line 95, in train
predictions = model(feature)
File "C:\Program Files\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "D:\nlp\code\Hierarchical_Attention_Networks_PyTorch\src\hierarchical_att_model.py", line 42, in forward
output, self.sent_hidden_state = self.sent_att_net(output, self.sent_hidden_state)
File "C:\Program Files\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "D:\nlp\code\Hierarchical_Attention_Networks_PyTorch\src\sent_att_model.py", line 37, in forward
output = matrix_mul(output, self.context_weight).permute(1, 0)
File "D:\nlp\code\Hierarchical_Attention_Networks_PyTorch\src\utils.py", line 39, in matrix_mul
feature = torch.mm(feature, weight)
RuntimeError: matrices expected, got 1D, 2D tensors at c:\programdata\miniconda3\conda-bld\pytorch-cpu_1532496521117\work\aten\src\th\generic/THTensorMath.cpp:2065
Does anyone know about it?
Thank you very much.