Hi everyone,
As a toy project, I coded a model to classify sentences whether they are more likely written by Lewis Carroll or Mary Shelley, so I fed my model the sentences of “Alice in Wonderland” and “Frankenstein” with the correct lablels.
As with every sequence problem, I had to pad shorter sentences with 0 (I used a maximum length of 30). Now comes the weird part: when I used nn.CrossEntropyLoss(), I achieved an accuracy of 89% on the test set. With nn.CrossEntropyLoss(ignore_index=0), my accuracy on the test set fell to 71%. I can’t think to myself why this would happen.
Here’s my model for reference:
class BookClassifier(nn.Module):
def __init__(self, vocab_size, embedding_dim=100, hidden_size=100, num_layers=2, dropout_p=0.2):
super().__init__()
self.embedding = nn.Embedding(vocab_size, embedding_dim, padding_idx=0)
self.bilstm = nn.LSTM(input_size=embedding_dim,
hidden_size=hidden_size,
num_layers=num_layers,
batch_first=True,
dropout=dropout_p,
bidirectional=True)
self.fc = nn.Linear(2 * hidden_size, 2)
def forward(self, x):
embedded = self.embedding(x)
out, _ = self.bilstm(embedded)
return self.fc(out[:, -1, :])
I used optim.Adam(model.parameters()) as an optimizer, and the usual training loop.
Any insights are appreciated 