Hacker News Reader

Comment by m0th87 | Hacker News Reader

Looks like it can even handle garden path sentences [1] like "the complex houses married and single soldiers and their families."

1: https://en.wikipedia.org/wiki/Garden_path_sentence

View on HN

You usually can't tell from one single sentence if a parser handles garden path sentences, which involve backtracking (hence the name).

Plus, there is a part-of-speech tagger that provides an intermediate layer of nlp analysis, which makes a backtracking involving a change of part-of-speech («houses» being either a verb of a noun) quite harder (yet not impossible).

POS taggers improve parsers performance dramatically but usually are an indication that the parser have limited backtracking capabilities.

You can try it with «the complex houses married soldiers», houses is erroneously marked as a noun.

Yup. It gets it fundamentally wrong, thinking "complex" is an adjective and "houses" is a noun:

  the/DT
  complex/JJ
  houses/NNS
  married/JJ
  and/CC
  single/JJ
  soldiers/NNS
  and/CC
  their/PRP$
  families/NNS

But, honest, that's a rotten way to try out NL parsers. Nobody claims Stanford's, or anyone's, parser can handle that sort of thing. It throws human language organs because it's specially engineered to do that. And if it can confuse humans, there's no program that it can't confuse, except for one where the right parse is hard-coded.

It's an unfair test, is what I'm saying.

Do any dependency parsers do this well? I think 'houses' throws them off, e.g.:

https://demos.explosion.ai/displacy/?text=The%20complex%20ho...

vs.

https://demos.explosion.ai/displacy/?text=The%20complex%20co...

"The horse raced past the barn fell" still trips it.

Try: Time flies like the wind but fruit flies like bananas.

I don't think this parser parses left-to-right, like humans and other parsers like Parsley MacParseface do.

It does. left-to-right linear time parsing is the idea that's about 15 years old and turned out to be most efficient and most performant.

I might be wrong, but based on the parser home page it looks to me like they provide a Shift-reduce parser but it is not the default. I even says on the page Parser: englishPCFG.ser.gz.

True, it might not be the case, although they do have the exact same beam search shift-reduce parser that mcparseface has (without the neural net).

I’m currently writing a paper on improving natural language interfaces for databases, and I’ve ran both Parsey McParseface and Stanford’s CoreNLP for that, and both have horrible performance.

Their results for even the most simple english sentences are horrible, often with multiple mistagged words.

As soon as you add relative clauses, they break down entirely.

Well, it's not really a solved problem, is it? Saying the performance is "horrible" does some injustice to it. It's actually the state of the art in solutions to a very difficult problem.

I've used the Stanford parser for university work (at Masters level). It was part of the pipeline for a sentiment extractor. I didn't get the feeling it performed badly, quite the contrary- the parses it generated were quite useful as features in my extractor. It had this rare feeling of a tool that you can actually use to do something interesting and useful.

Then again, I do have my expectations set very lowly, for this sort of thing, especially after completing my Masters thesis (on grammar induction). Language learning is hard.

I mean, it's surprising how well it works, but if you come in, and expect to get a perfect parse tree out of everything, so you can then easily pattern match on top of that, well, you'll be in for a rouch surprise.

Any every slightly more complicated sentence was completely misparsed by CoreNLP and ParseyMcParseface. As I mentioned before, as soon as you introduce relative clauses it breaks down.

It's hard when your research was supposed to discuss how to better handle the common knowledge problem of NLIDBs, but you have to spend a lot of time just to get a kinda useful parse out in the first place.

Reply on news.ycombinator.com