Getting the "TypeError: 'int' object is not subscriptable" error while constructing a basic LSTM network?

Hey everyone,

I’m getting the error below while constructing a basic LSTM network for research purposes (which is based on the source code available in this GitHub repository). Could you please help me to find out the issue?

p.s. I’m utilizing the Google Colab platform.

TypeError Traceback (most recent call last)

[<ipython-input-10-8883ef0b1eb5>]( in <module>() 195 196 if __name__ == '__main__': --> 197 main()

6 frames

[/usr/local/lib/python3.6/dist-packages/torch/nn/modules/]( in check_forward_args(self, input, hidden, batch_sizes) 520 expected_hidden_size = self.get_expected_hidden_size(input, batch_sizes) 521 --> 522 self.check_hidden_size(hidden[0], expected_hidden_size, 523 'Expected hidden[0] size {}, got {}') 524 self.check_hidden_size(hidden[1], expected_hidden_size,

TypeError: 'int' object is not subscriptable
import os
import random
import time

import torch
import torch.nn as nn
import torch.optim as optim
from import Dataset
from torchtext import data
from torchtext.datasets import IMDB

def epoch_time(start_time, end_time):
    elapsed_time = end_time - start_time
    elapsed_mins = int(elapsed_time / 60)
    elapsed_secs = int(elapsed_time - (elapsed_mins * 60))
    return elapsed_mins, elapsed_secs

class LSTM_IMDB(nn.Module):
    def __init__(self, input_dim, embedding_dim, hidden_dim, output_dim):
        self.hidden_dim = hidden_dim
        self.embedding = nn.Embedding(input_dim, embedding_dim)
        self.lstm = nn.LSTM(embedding_dim, hidden_dim)
        self.fc = nn.Linear(hidden_dim, output_dim)

    def forward(self, text):
        embedded = self.embedding(text)

        lstm_out, hidden=self.lstm(embedded, self.hidden_dim)

        #stack up the lstm output
        lstm_out=lstm_out.contiguous().view(-1, hidden)

        return self.fc(lstm_out)

def binary_accuracy(preds, y):
    Returns accuracy per batch, i.e. if you get 8/10 right, this returns 0.8, NOT 8

    # round predictions to the closest integer
    rounded_preds = torch.round(torch.sigmoid(preds))
    correct = (rounded_preds == y).float()  # convert into float for division
    acc = correct.sum() / len(correct)
    return acc

def train(model, iterator, optimizer, criterion):
    epoch_loss = 0
    epoch_acc = 0


    for batch in iterator:

        predictions = model(batch.text).squeeze(1)

        loss = criterion(predictions, batch.label)

        acc = binary_accuracy(predictions, batch.label)



        epoch_loss += loss.item()
        epoch_acc += acc.item()

    return epoch_loss / len(iterator), epoch_acc / len(iterator)

def evaluate(model, iterator, criterion):
    epoch_loss = 0
    epoch_acc = 0


    with torch.no_grad():
        for batch in iterator:
            predictions = model(batch.text).squeeze(1)

            loss = criterion(predictions, batch.label)

            acc = binary_accuracy(predictions, batch.label)

            epoch_loss += loss.item()
            epoch_acc += acc.item()

    return epoch_loss / len(iterator), epoch_acc / len(iterator)

def main():
    SEED = 1234
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

    torch.backends.cudnn.deterministic = True

    TEXT = data.Field(tokenize='spacy')
    LABEL = data.LabelField(dtype=torch.float)

    train_data, test_data = IMDB.splits(TEXT, LABEL)
    train_data, valid_data = train_data.split(split_ratio=0.5, random_state=random.seed(SEED))

    print(f'Number of training examples: {len(train_data)}')
    print(f'Number of validation examples: {len(valid_data)}')
    print(f'Number of testing examples: {len(test_data)}')

    MAX_VOCAB_SIZE = 25_000
    TEXT.build_vocab(train_data, max_size=MAX_VOCAB_SIZE)

    BATCH_SIZE = 64
    train_iterator, valid_iterator, test_iterator = data.BucketIterator.splits(
        (train_data, valid_data, test_data),

    INPUT_DIM = len(TEXT.vocab)
    HIDDEN_DIM = 256
    OUTPUT_DIM = 1


    lr = 1e-2
    criterion = nn.BCELoss()
    criterion =
    optimizer = optim.SGD(model.parameters(), lr=lr, momentum=0.9)

    N_EPOCHS = 5

    best_valid_loss = float('inf')

    for epoch in range(N_EPOCHS):

        start_time = time.time()

        train_loss, train_acc = train(model, train_iterator, optimizer, criterion)
        valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)

        end_time = time.time()

        epoch_mins, epoch_secs = epoch_time(start_time, end_time)

        if valid_loss < best_valid_loss:
            best_valid_loss = valid_loss
  , '')

        print(f'Epoch: {epoch + 1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s')
        print(f'\tTrain Loss: {train_loss:.3f} | Train Acc: {train_acc * 100:.2f}%')
        print(f'\t Val. Loss: {valid_loss:.3f} |  Val. Acc: {valid_acc * 100:.2f}%')

if __name__ == '__main__':

Could you check, if you are accidentally passing an int to the model instead of a tensor?
The error is raised, if you are trying to index an int via:

> TypeError: 'int' object is not subscriptable

which seems to be called somewhere inside the forward method.

1 Like

Hey @ptrblck,
Thanks for your interest. I don’t purposely do such thing as I use the iterator mechanism on a pre-defined, standard dataset, namely, IMDB. The error might be due to the confusion between the hidden and self.hidden_dim params located in the forward function. When you have some free time, could you please check them?

The error is raised in:

lstm_out, hidden=self.lstm(embedded, self.hidden_dim)

since you are passing in the self.hidden_dim, which is an integer instead of the expected hidden tensor.
Have a look at the docs for more information regarding its shape etc. or just remove it, if you want to use the default zero-initialized hidden.

Once this is fixed, you would run into another issue, as you are trying to use the returned hidden tensor to reshape the lstm_out in:

lstm_out=lstm_out.contiguous().view(-1, hidden)

so you might want to use self.hidden_dim for it.
That being said, note that this view operation would collapse the temporal and batch dimension.

1 Like

Thank you very much for your explanation, dear @ptrblck. In order to not collapse the temporal and batch dimension as a result of the view operation, now I don’t apply it. But, still I’m getting the ValueError: Target and input must have the same number of elements. target nelement (64) != input nelement (76800) error.

p.s. Here is the corrected version of the forward function:

def forward(self, text):
    embedded = self.embedding(text)

    lstm_out, hidden = self.lstm(embedded)

    return self.fc(lstm_out)

I guess 64 would be the batch size, while 76800 would probably be the temporal size (or the flattened batch dim and temp. dim)?

Anyway, I think you might want to consider using the last time step only or reducing the temporal dimension somehow (e.g. with a mean) to get an output of [batch_size, out_features].

I’m not familiar with your use case, so please feel free to add more information in case my suggestions wouldn’t work.

Exactly, 64 is the batch size, and 76,800 is the temporal size. But could not get what you mean by using the last time step only? Actually, I’ve added the whole source code in the OP. Could you please guide me on it?