In the forward propagation process, the feature changes from two-dimensional [btach_size and future_dim] to one-dimensional [future_dim], why?


(Jiangjian) #1

There is a pytorch implementation of the model described in the paper “Hierarchical Attention Networks for Document Classification” paper. in gihub. When I was using my own data, there was an error:In the forward propagation process, the feature changes from two-dimensional [btach_size and future_dim] to one-dimensional [future_dim].

43
fature: torch.Size([128, 200])
tensor([[-0.0094, -0.0273, -0.0221,  ...,  0.0299,  0.0019, -0.1354],
        [-0.0314, -0.0314, -0.0286,  ...,  0.0330,  0.0025, -0.0863],
        [-0.0149, -0.0274, -0.0402,  ...,  0.0417,  0.0114, -0.0832],
        ...,
        [-0.0280, -0.0274, -0.0387,  ...,  0.0508, -0.0074, -0.1086],
        [-0.0135, -0.0102, -0.0637,  ...,  0.0753,  0.0128, -0.0639],
        [-0.0204, -0.0274, -0.0380,  ...,  0.0388,  0.0185, -0.0865]])

44
fature: torch.Size([200])
tensor([ 0.0391,  0.0423, -0.0006,  0.0140, -0.0563, -0.0199,  0.0343,  0.0095,
         0.0262,  0.0202, -0.0553,  0.0087, -0.0835, -0.0059, -0.0274,  0.0213,
         0.0568, -0.0260,  0.0170,  0.0073,  0.0744,  0.0198,  0.0064,  0.0348,
        -0.0576,  0.0424, -0.0401, -0.0294,  0.0116,  0.0395,  0.0107, -0.0550,
        -0.1116,  0.0902, -0.0510,  0.0082, -0.0073,  0.0013,  0.0503, -0.0314,
         0.0340, -0.0393,  0.0026,  0.0085,  0.0219,  0.0207,  0.0008, -0.0266,
         0.0642,  0.0353,  0.0158, -0.0266, -0.0135, -0.0798,  0.0170,  0.0014,
         0.0501, -0.0407, -0.1198, -0.0407,  0.0132, -0.0408, -0.0646,  0.0879,
        -0.0117, -0.0062,  0.0437,  0.0289, -0.0196, -0.0293, -0.0125,  0.0805,
         0.0368, -0.0111, -0.0031, -0.0010,  0.0388, -0.0483, -0.0001,  0.0167,
        -0.0459, -0.0234, -0.0038,  0.0443,  0.0501, -0.0359, -0.0395, -0.0486,
         0.0451, -0.0089,  0.0270, -0.0063, -0.0079, -0.0057,  0.0067, -0.0185,
         0.0217,  0.0746, -0.0117,  0.0175, -0.0089, -0.0106, -0.0345, -0.0077,
        -0.0411, -0.0724,  0.0220, -0.0212, -0.0246,  0.0863,  0.0113, -0.0067,
         0.0016, -0.0506,  0.0203,  0.0210, -0.0541, -0.0059, -0.0213,  0.0214,
         0.0144, -0.0114,  0.0491, -0.0107, -0.0105,  0.0096, -0.0364, -0.0067,
        -0.0130,  0.0528, -0.0035, -0.0092,  0.0198,  0.0031, -0.0109,  0.0033,
         0.0193, -0.0492,  0.0038,  0.0780, -0.0177,  0.0428,  0.0174, -0.0387,
         0.0426, -0.0266, -0.0248,  0.0047, -0.0175, -0.0035,  0.0937,  0.0171,
        -0.0195,  0.0106,  0.0133,  0.0472,  0.0144, -0.0754, -0.0273, -0.0104,
        -0.0112,  0.0168,  0.0012,  0.0329,  0.0295,  0.0017, -0.0366,  0.0655,
         0.0358,  0.0519,  0.0016, -0.0390, -0.0407,  0.0162, -0.0045,  0.0121,
         0.0123,  0.0083,  0.0304,  0.0358,  0.0617, -0.0152, -0.0165, -0.0002,
        -0.0001, -0.0143, -0.0056,  0.0038,  0.0150,  0.0724,  0.0733,  0.0152,
         0.0667,  0.0588, -0.0420,  0.0514, -0.0170,  0.0184,  0.0426, -0.0109])

Then I got an error message.

Traceback (most recent call last):
  File "D:/nlp/code/Hierarchical_Attention_Networks_PyTorch/train.py", line 157, in <module>
    train(opt)
  File "D:/nlp/code/Hierarchical_Attention_Networks_PyTorch/train.py", line 95, in train
    predictions = model(feature)
  File "C:\Program Files\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "D:\nlp\code\Hierarchical_Attention_Networks_PyTorch\src\hierarchical_att_model.py", line 42, in forward
    output, self.sent_hidden_state = self.sent_att_net(output, self.sent_hidden_state)
  File "C:\Program Files\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "D:\nlp\code\Hierarchical_Attention_Networks_PyTorch\src\sent_att_model.py", line 37, in forward
    output = matrix_mul(output, self.context_weight).permute(1, 0)
  File "D:\nlp\code\Hierarchical_Attention_Networks_PyTorch\src\utils.py", line 39, in matrix_mul
    feature = torch.mm(feature, weight)
RuntimeError: matrices expected, got 1D, 2D tensors at c:\programdata\miniconda3\conda-bld\pytorch-cpu_1532496521117\work\aten\src\th\generic/THTensorMath.cpp:2065

Does anyone know about it?

Thank you very much.


(Jiangjian) #2

I have solved it. It was my data that caused the error.