Stanford Natural Language Parser | Hacker News Reader

Stanford Natural Language Parser

nlp.stanford.edu

142 pointsaphextron65 comments

Thanks to Sergey Tihon in Minsk the Stanford NLP library gets made available as a .NET library about as fast as new releases come out. http://sergey-tihon.github.io/Stanford.NLP.NET/

Just checked that out. It looks fantastic, big thanks for sharing.

I haven't seen too many things for NLP on .NET. I have run across a ton things on Python or C++/C. Would love to see more inn .NET.

My favourite example phrase to mess with NL parsers:

Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo

Stanford fails miserably on this one, even at the tagging step:

  Buffalo/NNP
  buffalo/NNP
  Buffalo/NNP
  buffalo/JJ
  buffalo/JJ
  buffalo/NN
  Buffalo/NNP
  buffalo/NNP

... but then, most humans also find that sentence really hard to parse.

Check it out:

https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffal...

That's great. But let me ask, technically, how is the sentence "a grammatically correct sentence in American English" without punctuation? For NLP, shouldn't this first be regarded as a 'special case' of unpunctuated sentences? (and BTW, the parser doesn't handle the punctuated version any better).

It is in fact grammatically correct without punctuation, just somewhat awkward. Consider this analogous sentence with the same meaning, with optional parentheticals added to help you parse:

New York oxen (that) New York oxen bully (themselves) bully New York oxen.

the fact that nobody can understand without a preposition and a pronoun even after you replaced the uncommon words with common ones tells me this is a bunch a buffalo excrement that barely entertain linguist phds.

That sentence is virtually the same parse tree as "The man I saw crossed the street," which is a totally reasonable sentence to most English speakers. The difference in the parsing is that the "I" in this case corresponds to "Buffalo buffalo", a noun phrase--and the sentence is confusing because we have a tendency to add the "that" for noun phrases instead of single-word noun/pronouns, which the sentence is missing.

For me it was confusing the first time I read it because I didn't know that "buffalo" can mean "to bully". Without that information it doesn't make sense at all.

So I never thought there's something inherently difficulty in the syntax of the sentence- I thought it's just that unexpected third use of "buffalo" that makes it so strange.

Of course, now that I think of it, what you say makes sense also. For an NL parser, the variant use of "buffalo" shouldn't be an issue. You could even say that lacking the context of what a "buffalo" really is, a parser should even find it easier to parse the sentence than a human, who must pause to try and understand what it means.

Maybe it's not such a bad sentence to test parsers after all.

'Buffalo' doesn't mean 'bully' in English. It means 'con.'

The Wikipedia article above says that though uncommon in usage, the buffalo verbs in that sentence mean "to bully, harass, or intimidate" or "to baffle".

AIUI, it's only "to bully" in the sense of verbally overwhelming someone into believing what you what them to believe. It's like the word "to snow" (in the sense that means "to con").

I would read "buffaloing" somebody as a less vulgar version of bullshitting, which is also a way of lying about something, mostly to get them to buy something they don't actually need.

I don't think "buffalo" (the verb) having a specific meaning makes any difference in how hard that sentence is to parse.

Crucially, what is a reasonable sentence depends heavily on words.

"History lamps traffic people house cry Michigan algebra" also has the same parse tree, but if I utter this, people will probably think I need medical attention.

I doubt most english speakers would correctly parse this sentence either, so I'd say the results are human-like? :)

Most English speakers can parse the sentence just fine when they hear it spoken.

the argument is nevertheless valid: the parser does not process any additional audible signal

I would not have been able to tell which buffalos were verbs, which were nouns and which were proper nouns without the rephrasing of the sentence on Wikipedia. In fact even with that rephrasing I still find it difficult to tell which buffalo in the original sentence is what.

Not so. I've tried this on a few dozen students (post-secondary, many of them post-baccalaureate, perhaps 50% ESL) over the years; of those who have not heard it previously, all others say "what? that's not a sentence". I don't recall a single counter-example.

That's surprising to hear. I don't think I've ever found someone that understood it without an explanation. I like using it as an example of how confusing English can be to parse.

English is tough, though through thorough thought it can be understood.

Shouldn't that be "English is tough, though through thorough thought it can be taught"?

I just found out that Alexa has a funny response to this sentence.

Funny, but jokes aside, I found the NLP to be quite impressive / accurate.

I put the following phrase

> I nut on my girlfriends face.

and the parser correctly tagged 'nut' as a VBP (i.e. Verb, non-3rd Person Singular resent) i.e. that 'nut' in this context is a verb, and not a noun.

Here's what the tagging returned (all words with tags)

I/PRP nut/VBP on/IN my/PRP$ girlfriends/JJ face/NN ./.

Tag Descriptions: http://stackoverflow.com/a/1833718/325521

I don't know if that sentence is a grammatical correct sentence in English. Moreover, I don't know what the meaning of the sentence.

I wonder how the other NLP tools parse that sentence. These are the results:

* Spacy [1]

  Buffalo/NNP
  buffalo/NN
  Buffalo/NNP
  buffalo/NN
  buffalo/NN
  buffalo/NN
  Buffalo/NNP
  buffalo/NN

* NLP4J [2]

  Buffalo/NNP
  buffalo/NNP
  Buffalo/NNP
  buffalo/NNP - pos2=CD
  buffalo/NNP - pos2=CD
  buffalo/NNP
  Buffalo/NNP
  buffalo/NNP

Well, that sentence really makes NLP tools confused.

[1]: https://spacy.io/

[2]: https://emorynlp.github.io/nlp4j/

Parser home: http://nlp.stanford.edu/software/lex-parser.shtml

My undergraduate research used Stanford Parser for getting word tags in a sentence before calculating its similarity with another sentence. Stanford Parser results are good, but it fails in some sentences. If you need a better alternative than this (for JVM languages), I would recommend NLP4J [1].

[1]: https://emorynlp.github.io/nlp4j/

Not sure if this is the right thread to ask, wanted to ask in https://news.ycombinator.com/item?id=13445255 but I missed the boat:

How difficult is it to use NLP to do keyword extraction for non-English languages? Any pointer to get started? (not NLP in general, but how to tailor it for non-English)

As long as you have a tokenizer, i.e. you can split words in a sentence, language doesn't matter for some basic techniques.

Two examples that easily come to mind: TF-IDF [1] and TextRank [2,3].

These are good to get started, but if you want to know more about the state of the art, searching on Google Scholar for "keyword extraction $language" is your best bet (and maybe "keyword extraction overview", also in Scholar).

[1] http://www.joyofdata.de/blog/tf-idf-statistic-keyword-extrac...

[2] http://digital.library.unt.edu/ark%3A/67531/metadc30962/m2/1...

[3] https://github.com/davidadamojr/TextRank

Keyword extraction is not easy for other languages, but POS tagging and parsing is easy. There's a project called UDPipe which has pretrained models for most languages. That's the best "general" solution I've found.

It is a nice statistical parser. I did note that it does not do anaphora resolution (map pronouns to original referred to nouns, etc.)

did you try http://nlp.stanford.edu/projects/coref.shtml

Thanks, I have tried that online but it is not available as a library. My kbsportal library does collocation, but not well.

I always thought http://corenlp.run was a better demo of the tool. Sadly it's no longer available: https://github.com/stanfordnlp/CoreNLP/issues/273

You could use this Dockerfile to run a local copy if you're interested: https://gist.github.com/charlieegan3/910276eef0f8658b44b42af...

Looks like it can even handle garden path sentences [1] like "the complex houses married and single soldiers and their families."

1: https://en.wikipedia.org/wiki/Garden_path_sentence

Do any dependency parsers do this well? I think 'houses' throws them off, e.g.:

https://demos.explosion.ai/displacy/?text=The%20complex%20ho...

vs.

https://demos.explosion.ai/displacy/?text=The%20complex%20co...

I don't think this parser parses left-to-right, like humans and other parsers like Parsley MacParseface do.

It does. left-to-right linear time parsing is the idea that's about 15 years old and turned out to be most efficient and most performant.

I might be wrong, but based on the parser home page it looks to me like they provide a Shift-reduce parser but it is not the default. I even says on the page Parser: englishPCFG.ser.gz.

True, it might not be the case, although they do have the exact same beam search shift-reduce parser that mcparseface has (without the neural net).

I’m currently writing a paper on improving natural language interfaces for databases, and I’ve ran both Parsey McParseface and Stanford’s CoreNLP for that, and both have horrible performance.

Their results for even the most simple english sentences are horrible, often with multiple mistagged words.

As soon as you add relative clauses, they break down entirely.

Well, it's not really a solved problem, is it? Saying the performance is "horrible" does some injustice to it. It's actually the state of the art in solutions to a very difficult problem.

I've used the Stanford parser for university work (at Masters level). It was part of the pipeline for a sentiment extractor. I didn't get the feeling it performed badly, quite the contrary- the parses it generated were quite useful as features in my extractor. It had this rare feeling of a tool that you can actually use to do something interesting and useful.

Then again, I do have my expectations set very lowly, for this sort of thing, especially after completing my Masters thesis (on grammar induction). Language learning is hard.

I mean, it's surprising how well it works, but if you come in, and expect to get a perfect parse tree out of everything, so you can then easily pattern match on top of that, well, you'll be in for a rouch surprise.

Any every slightly more complicated sentence was completely misparsed by CoreNLP and ParseyMcParseface. As I mentioned before, as soon as you introduce relative clauses it breaks down.

It's hard when your research was supposed to discuss how to better handle the common knowledge problem of NLIDBs, but you have to spend a lot of time just to get a kinda useful parse out in the first place.

"The horse raced past the barn fell" still trips it.

You usually can't tell from one single sentence if a parser handles garden path sentences, which involve backtracking (hence the name).

Plus, there is a part-of-speech tagger that provides an intermediate layer of nlp analysis, which makes a backtracking involving a change of part-of-speech («houses» being either a verb of a noun) quite harder (yet not impossible).

POS taggers improve parsers performance dramatically but usually are an indication that the parser have limited backtracking capabilities.

You can try it with «the complex houses married soldiers», houses is erroneously marked as a noun.

Yup. It gets it fundamentally wrong, thinking "complex" is an adjective and "houses" is a noun:

  the/DT
  complex/JJ
  houses/NNS
  married/JJ
  and/CC
  single/JJ
  soldiers/NNS
  and/CC
  their/PRP$
  families/NNS

But, honest, that's a rotten way to try out NL parsers. Nobody claims Stanford's, or anyone's, parser can handle that sort of thing. It throws human language organs because it's specially engineered to do that. And if it can confuse humans, there's no program that it can't confuse, except for one where the right parse is hard-coded.

It's an unfair test, is what I'm saying.

Try: Time flies like the wind but fruit flies like bananas.

Does anyone here use parsers and know of practical applications for them?

My main use case is unsupervised feature extraction from blob text. One application may be to extract all noun entities from a news article, so you can automatically figure out who the article is about.

Good idea! However, chunking (shallow parsing) might be enough for that purpose.

Features you can use for other stuff, like entity detection.

E.g. AlchemyAPI has a cool demo, where they can detect names of diseases, cities, etc. I suspect the parts of speech would improve the accuracy for tagging words that have multiple meanings.

I used the disease bit on AlchemyAPI for https://www.findlectures.com, to make facets for a health topics: https://www.findlectures.com/?p=1&class1=Science&category_l2...

I use this parser in particular at work designing a closed-domain chatbot.

Do you mind going into more detail, since most chatbots just use rules and keyword words.

Taking the opportunity to plug my own D3 visualization using CoreNLP : )

http://nlpviz.bpodgursky.com/

The assumptions we make, the more performance goes down in tasks like machine translation, and that includes the assumption that parts of speech matter to machines.

Ah, but can it tell when I'm trying to be witty?

That's not its job.

I understand that -- as a parser, of course it's not its job to understand sarcasm or wit.

Just referencing the old Chomskian bit about statistical correlates and how relevant it is to understanding the fundamental principles behind the expression of language (rather than merely its syntax).

I guess it was a bad joke gone over. ah well

I really wanted to use the .NET port of this, but unfortunately it is GPL... dead in the water for commercial software...

Previous discussion: https://news.ycombinator.com/item?id=11273240

Unfortunately with zero comments.

This software kinda sucks. I guess you get what you pay for, sometimes.

It is crazy slow and uses absurd amounts of memory, as well as being simultaneously an object lesson in both over-engineering, and under-engineering. Built by academics, so it's not unexpected.

Nice example of the application of recent NLP thesis topics to parsing problems, though. Quite suitable as documentation and reference for, e.g. future work on SpaCy - which is much more usable in productive contexts.

Yes, it does use a reasonable chunk of memory (I wouldn't run in with less than 3GB available)

However, speed wise, its only slow in my experience, if you try to integrate via a shell script - if you run it as a service and make http calls it's been amazingly fast in my usage.

I believe they include a "service" mode as part of the package now, but for a project a little while ago I forked & extended a project that provides a JSON/XML http service: https://github.com/Koalephant/StanfordCoreNLPHTTPServer

In my testing it struck a good balance between accuracy and performance. I was also impressed by the lengths the group goes to help people on their mailing list. That said, their licensing is not friendly so I generally have to rule it out.

Generic sentence from newspaper article. I heard Stanford uses Java?

  (ROOT
    (S
      (PP (IN In)
        (S
          (VP (VBG keeping)
            (PP (IN with)
              (NP (PRP$ his) (JJ insurgent) (NN campaign))))))
      (, ,)
      (NP (NNP President) (NNP Trump))
      (VP
        (VP (VBD dispensed)
          (PP (IN with)
            (NP (NNS appeals)))
          (PP (TO to)
            (NP (NN unity))))
        (CC or)
        (VP (VBZ attempts)
          (S
            (VP (TO to)
              (VP (VB build)
                (NP (NNS bridges))
                (PP (TO to)
                  (NP
                    (NP (PRP$ his) (NNS opponents))
                    (PP (IN in)
                      (NP (PRP$ his) (JJ inaugural) (NN address))))))))))
      (. .)))

Reply on news.ycombinator.com