I want to build a model in Pytorch where I have text as input. Each batch consists of multiple text passages which consist of words, which in turn consist of characters. Additionally to using word embeddings I would like to process character embeddings through an bidirectional RNN to generate on character-based embedding per word to help with out-of-vocab words.
How do I process the character embeddings of size [batch_size, num_words, num_chars, char_embedding_size] through an RNN? There is one dimension too much and when I do char_embeddings.view([batch_size*num_words, num_chars, char_embedding_size]) I cannot create a packed sequence as the lengths of the words are not ordered.
How can I still process these char embeddings through a bidirectional RNN while still correctly handling the different number of characters in each word?