LSTM time step without loop

I was reading Seq2seq. The below code is for getting output from LSTM for various time steps. Is it possible to eliminate this loop and get the output in one line?

    for ei in range(input_length):
        encoder_output, encoder_hidden = encoder(
            input_tensor[ei], encoder_hidden)
        encoder_outputs[ei] = encoder_output[0, 0]
1 Like

Look at how the forward() method of Encoder is implemented. it embeds a single vector, and then runs it through the GRU (rather than LSTM here) stored on that class. the pytorch GRU and LSTM classes can, by default take a sequence in the form of [batch_length, sequence_length, embedding_dim]. You can therefore embed a sequence of input_length to be a tensor with dimensions [1, sequence_length, embedding_dim], and run that through the LSTM in one go, allowing the efficient pytorch code to iterate over it, rather than a python loop. You can even batch your sequences and embed those, and pass them through an LSTM or GRU.

1 Like

So in batch case how would you ensure that decoder terminates when EOS token is an output of current time step?
Since in loop case we can know the output at each time step and stop when EOS is output as below:

            topv, topi = decoder_output.topk(1)
            decoder_input = topi.squeeze().detach() 
            if decoder_input.item() == EOS_token:
                break

Hi, Above you have the encoder, which can be done without the loop.

For decoding, I don’t think you can simply pass an entire sequence, because you need to make a prediction at each timestep before moving to the next one. The RNN is informed by both the encoder, and the previously predicted item in the sequence. You can, of course, still use a batch, just not a batch of an entire sequence.

E.g. a tensor of (batch_size, 1, dims) should be doable.

2 Likes