Why pytorch doesn't offer attention or crf api (like nn module)?

nlp
1

Hi, I’m a big fan of pytorch and nlp researcher.

Last year, I was struggle to implement attention and crf layers for sequence labeling task.

Compared to TensorFlow, I think rnn modules are not support well. (in TF, attention or crf can be used through just one line)

Although pytorch offers attention and crf tutorial, as I know it doesn’t suitable for batch.

I wonder is there any plan to offer official apis for attention and crf like TF.

1 Like
2

You can find an implementation of Attention in PyTorch-NLP. I also have implemented the attention mechanism for the hierarchical attention network based on Yang et al. 2016: https://gist.github.com/notnami/a3b4e2e7ff127121c937020c5cd87ace. I don’t know if my implementation is the most efficient, but it has worked well in my experiments.