We’ve just open-sourced torchMoji, a pytorch implementation of DeepMoji (https://deepmoji.mit.edu/), a state-of the-art emotion/sentiment/sarcasm detection model trained on 1.6 billions tweets (!) that was published this summer by the MIT Media Lab.
I wrote a companion post (https://medium.com/huggingface/understanding-emotions-from-keras-to-pytorch-3ccb61d5a983) talking about the reimplementation of the original Keras model in pyTorch.
In the post I write some thoughts and code examples about
- how to make a custom pyTorch LSTM cell with custom activation functions,
- how the PackedSequence object works,
- comparing the implementation of an attention layer in Keras and pyTorch,
- our implementation of a smart BatchSampler that can sample in a balanced way from the unbalanced datasets and gather batches in epochs of fixed size, and
- weights initialization from Keras to pyTorch.
Hope you like it,