Lstm training accuarcy stays same

Hu_Penglong · July 3, 2019, 12:57pm

I am new to Pytorch. Here is my problem, I train a lstm classifier
my training precision don’t increase in a tiny dataset.
the classifier seems to guess the max num of label of the dataset for all the example.
epoch 0 total_loss = 1.409505
train data precision:58/100 = 58.00%
valid data precision:92/401 = 22.94%
save model
epoch 1 total_loss = 1.146850
train data precision:58/100 = 58.00%
valid data precision:92/401 = 22.94%
epoch 2 total_loss = 1.060287
train data precision:58/100 = 58.00%
valid data precision:92/401 = 22.94%
epoch 3 total_loss = 1.019745
train data precision:58/100 = 58.00%
valid data precision:92/401 = 22.94%
epoch 4 total_loss = 0.997164
train data precision:59/100 = 59.00%
valid data precision:92/401 = 22.94%
save model
epoch 5 total_loss = 0.983069
train data precision:59/100 = 59.00%
valid data precision:92/401 = 22.94%
epoch 6 total_loss = 0.973470
train data precision:59/100 = 59.00%
valid data precision:92/401 = 22.94%
epoch 7 total_loss = 0.966532
train data precision:59/100 = 59.00%
valid data precision:92/401 = 22.94%
epoch 8 total_loss = 0.961307
train data precision:59/100 = 59.00%
valid data precision:92/401 = 22.94%
here is my code.

github.com

lonelydancer/intent/blob/master/intent.py

import sys
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader
from torch.nn.utils.rnn import pad_sequence, pack_padded_sequence

train_filename = '../data/a'
#train_filename = '../data/train.data'
#train_filename = '../data/intent.data'
valid_filename = '../data/valid.data'
batch_size = 128
#batch_size = 16
input_dim = 128
hidden_dim = 64
train_model = sys.argv[1]
save_path = '../model/model.test'
epoch_num = 200

This file has been truncated. show original

tom · July 3, 2019, 2:10pm

PyTorch RNNs take seq,batch,feature inputs unless you instantiate with batch_first=True. The full details are in the documentation.

Best regards

Thomas

Hu_Penglong · July 4, 2019, 3:38am

@tom
my training precision don’t increase in a tiny dataset.
the classifier seems to guess the max num of label of the dataset for all the example.
epoch 0 total_loss = 1.409505
train data precision:58/100 = 58.00%
valid data precision:92/401 = 22.94%
save model
epoch 1 total_loss = 1.146850
train data precision:58/100 = 58.00%
valid data precision:92/401 = 22.94%
epoch 2 total_loss = 1.060287
train data precision:58/100 = 58.00%
valid data precision:92/401 = 22.94%
epoch 3 total_loss = 1.019745
train data precision:58/100 = 58.00%
valid data precision:92/401 = 22.94%
epoch 4 total_loss = 0.997164
train data precision:59/100 = 59.00%
valid data precision:92/401 = 22.94%
save model
epoch 5 total_loss = 0.983069
train data precision:59/100 = 59.00%
valid data precision:92/401 = 22.94%
epoch 6 total_loss = 0.973470
train data precision:59/100 = 59.00%
valid data precision:92/401 = 22.94%
epoch 7 total_loss = 0.966532
train data precision:59/100 = 59.00%
valid data precision:92/401 = 22.94%
epoch 8 total_loss = 0.961307
train data precision:59/100 = 59.00%
valid data precision:92/401 = 22.94%
here is my code.

github.com

lonelydancer/intent/blob/master/intent.py

import sys
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader
from torch.nn.utils.rnn import pad_sequence, pack_padded_sequence

train_filename = '../data/a'
#train_filename = '../data/train.data'
#train_filename = '../data/intent.data'
valid_filename = '../data/valid.data'
batch_size = 128
#batch_size = 16
input_dim = 128
hidden_dim = 64
train_model = sys.argv[1]
save_path = '../model/model.test'
epoch_num = 200

This file has been truncated. show original

DoubtWang · July 5, 2019, 12:58pm

I have met this problem,
and my reason is the shape of output.
You can check your code.

Hu_Penglong · July 7, 2019, 7:14am

i think my output shape is ok. (batch_num, cls_num)

Hu_Penglong · July 11, 2019, 9:52am

my learning rate is too large.
thank you~