Why is reindexing a Dataset object causing CUDA memory errors

I have a pretty simple Dataset object which loads short videos (400 frames about 512*780 pixels each) from memory.
I have the following lines of code. Notice I index my global_dataset object twice at the same index. Unless I remove the latter indexing operation (last line of code), I get a CUDA memory error regardless of batch size (always 1 for this application) and epochs of training on a small recurrent CNN model. Why might this be? My Dataset object uses torchvision.IO to load the videos and is very similar to the ones in official Pytorch tutorials. Please let me know if more information would be useful.

with torch.no_grad():
        one_left_out, label = global_dataset[index]
 
        logits = net(one_left_out)
        
        last = logits[len(logits)-1]
        last = last.unsqueeze(0)
        label = label.unsqueeze(0)
        
        loss = loss_func(last, label)
 
        _, exp = global_dataset[index]

Can you share the error and maybe have a look at how much memory is used without the last line ( using nvidia-smi)