LSTM input and output dimentions

nlp
1

I am a bit confused about LSTM input and output dimensions

Here is my network:
Intent_LSTM(
  (embedding): Embedding(41438, 400)
  (lstm): LSTM(400, 512, num_layers=2, batch_first=True, dropout=0.5)
  (dropout): Dropout(p=0.5, inplace=False)
  (fc): Linear(in_features=512, out_features=3, bias=True)
).

Also this is my forward pass:

def forward(self, x):
        """
        Perform a forward pass

        """
        batch_size = x.size(0)

        x = x.long()
        embeds = self.embedding(x)

        lstm_out, hidden = self.lstm(embeds)

    
        # stack up lstm outputs
        lstm_out = lstm_out.contiguous().view(-1, self.hidden_dim)


        out = self.dropout(lstm_out)
        out = self.fc(out)


        
        # reshape to be batch_size first
        out = out.view(batch_size, -1,3)
        #print("sig_out",sig_out.shape)
        out = out[:, -1,:] # get last batch of labels
        
        # return last sigmoid output and hidden state
        return out

Here the shape of my embdeddings is : [50, 150, 400] 50 being batch size, 150 seq lenth of my input. 400 being my embedded dimensions. I am feeding this into my LSTM. But when I was going through pytorch documentation. It states that input has to be in the form :

**input** of shape (seq_len, batch, input_size)

So should the input be converted to this format. ([150,50,400]) ?

If yes how do I do that?

2

You define your LSTM with batch_first=True. This changes the expected dimensions if the input to (batch, seq_len, input_size). Try setting batch_first=False, and you will see that an error is thrown due to incorrect dimensions.