[Project] PyTorch implementation of an emotion/sentiment/sarcasm detection model - pre-trained on 1.6 billion tweets

1

Hi,

We’ve just open-sourced torchMoji, a pytorch implementation of DeepMoji (https://deepmoji.mit.edu/), a state-of the-art emotion/sentiment/sarcasm detection model trained on 1.6 billions tweets (!) that was published this summer by the MIT Media Lab.

I wrote a companion post (https://medium.com/huggingface/understanding-emotions-from-keras-to-pytorch-3ccb61d5a983) talking about the reimplementation of the original Keras model in pyTorch.

In the post I write some thoughts and code examples about

  • how to make a custom pyTorch LSTM cell with custom activation functions,
  • how the PackedSequence object works,
  • comparing the implementation of an attention layer in Keras and pyTorch,
  • our implementation of a smart BatchSampler that can sample in a balanced way from the unbalanced datasets and gather batches in epochs of fixed size, and
  • weights initialization from Keras to pyTorch.

Hope you like it,

Happy pyCoding :slight_smile:

Thom

4 Likes
2

That’s great!

I’m looking forward to an automatic emoji-corrector mode on messenger (too many times I misunderstood my girlfriend’s posts and picked the wrong emoji)

Are you using the same data than in the paper?

3

Ha ha ha I can relate to that problem :grinning:

Yes, we are actually using the same pre-trained weights that were trained on the billions tweets.

Transferring weights from Keras to pyTorch is actually very simple (could have written a word or two about that indeed)