4 Dimension input to LSTM

Hi,

My input is of shape (batch_size * num_sentences * sentence * embeddings). Any help on how to pass this 4-d input to an LSTM ? As far as i know, LSTM only accepts inputs of dimension 3.

Thanks in advance

Depending on what constraints you have on the dependence between sentences, you could have

  1. Independent sentences: Each sentence modeled individually. (batch_size*num_sentences, sentence, embeddings)
  2. Independent paragraphs: Each paragraph(sequence of sentences) modeled independently. (batch_size, num_sentences*sentence, embeddings)

Alternatively, you could have two layers of LSTM, one modeling and embedding individual sentences, the second taking these sentence embeddings as input and modeling paragraphs

4 Likes