IndexError: index 64 is out of bounds for dimension 0 with size 64

PyTorch Live
1

I’ve been stuck here for a while now. Any help.

batch_size = 64

The error occurs here.( label = label[sorted_idx] )

def train(args, data_loader, model):

    criterion = torch.nn.CrossEntropyLoss()
    optimizer = torch.optim.Adam(model.parameters(), lr=args.learning_rate)
    min_loss = np.Inf
    
    for epoch in range(args.num_epochs):
        train_losses = [] 
        train_acc = 0.0
        total=0
        
        print(f"[Epoch {epoch+1} / {args.num_epochs}]")
        
        model.train()
        
        for i, (text, label) in enumerate(tqdm(data_loader)):
            model.eval()

            text = text.to(args.device)
            label = label.to(args.device)         
            
            input_lengths = torch.LongTensor([torch.max(text[i, :].nonzero()) +1 for i in range(text.size(0))])
            
            input_lengths, sorted_idx = input_lengths.sort(0, descending=True)
            # print(len(input_lengths), len(sorted_idx)) 
            # print(input_lengths,"\n", sorted_idx)
            text = text[sorted_idx]
            # print("text pass")
            label = label.squeeze()
            label = label[sorted_idx]
            # print("label pass")
            
            optimizer.zero_grad()

            output = model(text, input_lengths)
            
            loss = criterion(output, label)
            
            loss.backward()
            optimizer.step()

            train_losses.append(loss.item())
            total += label.size(0)

            train_acc += acc(output, label)

        epoch_train_loss = np.mean(train_losses)
        epoch_train_acc = train_acc/total
        print(f'Epoch {epoch+1}') 
        print(f'train_loss : {epoch_train_loss}')
        print('train_accuracy : {:.3f}'.format(epoch_train_acc*100))
2

label seems to have a shape of [64] so the sorted_idx tensor should contain values in the range [0, 63]. Using an index of 64 will create the out-of-bounds indexing errors so make sure the sorted_idx contains valid values.

3

You were right. So I tried changing the code, but there is the same error, is there a problem with my code?

input_lengths = torch.LongTensor([torch.max(text[i, :].nonzero()) +1 for i in range(text.size(0) if text.size(0) < args.batch_size - 1 else args.batch_size - 1 )])

[Epoch 2 / 5]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉| 2226/2227 [01:05<00:00, 33.94it/s]
Traceback (most recent call last):
  File "/Users/ibyeong-gwon/Downloads/RNN_project 2/train.py", line 162, in <module>
    train(args, train_loader, model)
  File "/Users/ibyeong-gwon/Downloads/RNN_project 2/train.py", line 92, in train
    label = label[sorted_idx]
IndexError: index 36 is out of bounds for dimension 0 with size 36
4

Yes, the issue is raised as label's shape doesn’t fit the max. index in your code, while it works for text.
Check the shapes of these two tensors (label and text) and check why you are expecting them to be the same and why that’s not the case.