NLP-progress: Tracking Progress in Natural Language Processing

nlpprogress.com

75 pointssndean3 comments

I am a little worried about the accuracy of sentiment classifiers, relative to how often I see them used and in applied research settings. They work well for binary classification on Tweets and IMDB reviews and not much else (I guess they do alright at 93% if you magically have a parse tree).

It's not just sentiment classification. Many of these NLP methods overfit to the minutiae of the particular SOTA data set and data type.

Language identification, NER, POS tagging, parsing, tokenization… you name it. Solved in theory, trouble in practice.

Using the algos on a concrete project, with a concrete "upstream" goal, is the only way to assess the gap between SOTA and your needs. Don't blindly trust these 90+% scores.

Great resource. How times change: I didn’t see any non-deep learning systems in this list. I had been out of the field of NLP for a while and then went to NAACL 2016 - very surprised to hear all of the deep learning papers.

Reply on news.ycombinator.com