Hacker News Reader

Comment by YeGoblynQueenne | Hacker News Reader

My favourite example phrase to mess with NL parsers:

Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo

Stanford fails miserably on this one, even at the tagging step:

  Buffalo/NNP
  buffalo/NNP
  Buffalo/NNP
  buffalo/JJ
  buffalo/JJ
  buffalo/NN
  Buffalo/NNP
  buffalo/NNP

... but then, most humans also find that sentence really hard to parse.

Check it out:

https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffal...

View on HN

I doubt most english speakers would correctly parse this sentence either, so I'd say the results are human-like? :)

Most English speakers can parse the sentence just fine when they hear it spoken.

Not so. I've tried this on a few dozen students (post-secondary, many of them post-baccalaureate, perhaps 50% ESL) over the years; of those who have not heard it previously, all others say "what? that's not a sentence". I don't recall a single counter-example.

That's surprising to hear. I don't think I've ever found someone that understood it without an explanation. I like using it as an example of how confusing English can be to parse.

English is tough, though through thorough thought it can be understood.

Shouldn't that be "English is tough, though through thorough thought it can be taught"?

I would not have been able to tell which buffalos were verbs, which were nouns and which were proper nouns without the rephrasing of the sentence on Wikipedia. In fact even with that rephrasing I still find it difficult to tell which buffalo in the original sentence is what.

the argument is nevertheless valid: the parser does not process any additional audible signal

That's great. But let me ask, technically, how is the sentence "a grammatically correct sentence in American English" without punctuation? For NLP, shouldn't this first be regarded as a 'special case' of unpunctuated sentences? (and BTW, the parser doesn't handle the punctuated version any better).

It is in fact grammatically correct without punctuation, just somewhat awkward. Consider this analogous sentence with the same meaning, with optional parentheticals added to help you parse:

New York oxen (that) New York oxen bully (themselves) bully New York oxen.

the fact that nobody can understand without a preposition and a pronoun even after you replaced the uncommon words with common ones tells me this is a bunch a buffalo excrement that barely entertain linguist phds.

That sentence is virtually the same parse tree as "The man I saw crossed the street," which is a totally reasonable sentence to most English speakers. The difference in the parsing is that the "I" in this case corresponds to "Buffalo buffalo", a noun phrase--and the sentence is confusing because we have a tendency to add the "that" for noun phrases instead of single-word noun/pronouns, which the sentence is missing.

Crucially, what is a reasonable sentence depends heavily on words.

"History lamps traffic people house cry Michigan algebra" also has the same parse tree, but if I utter this, people will probably think I need medical attention.

For me it was confusing the first time I read it because I didn't know that "buffalo" can mean "to bully". Without that information it doesn't make sense at all.

So I never thought there's something inherently difficulty in the syntax of the sentence- I thought it's just that unexpected third use of "buffalo" that makes it so strange.

Of course, now that I think of it, what you say makes sense also. For an NL parser, the variant use of "buffalo" shouldn't be an issue. You could even say that lacking the context of what a "buffalo" really is, a parser should even find it easier to parse the sentence than a human, who must pause to try and understand what it means.

Maybe it's not such a bad sentence to test parsers after all.

'Buffalo' doesn't mean 'bully' in English. It means 'con.'

The Wikipedia article above says that though uncommon in usage, the buffalo verbs in that sentence mean "to bully, harass, or intimidate" or "to baffle".

AIUI, it's only "to bully" in the sense of verbally overwhelming someone into believing what you what them to believe. It's like the word "to snow" (in the sense that means "to con").

I would read "buffaloing" somebody as a less vulgar version of bullshitting, which is also a way of lying about something, mostly to get them to buy something they don't actually need.

I don't think "buffalo" (the verb) having a specific meaning makes any difference in how hard that sentence is to parse.

I don't know if that sentence is a grammatical correct sentence in English. Moreover, I don't know what the meaning of the sentence.

I wonder how the other NLP tools parse that sentence. These are the results:

* Spacy [1]

  Buffalo/NNP
  buffalo/NN
  Buffalo/NNP
  buffalo/NN
  buffalo/NN
  buffalo/NN
  Buffalo/NNP
  buffalo/NN

* NLP4J [2]

  Buffalo/NNP
  buffalo/NNP
  Buffalo/NNP
  buffalo/NNP - pos2=CD
  buffalo/NNP - pos2=CD
  buffalo/NNP
  Buffalo/NNP
  buffalo/NNP

Well, that sentence really makes NLP tools confused.

[1]: https://spacy.io/

[2]: https://emorynlp.github.io/nlp4j/

I just found out that Alexa has a funny response to this sentence.

Funny, but jokes aside, I found the NLP to be quite impressive / accurate.

I put the following phrase

> I nut on my girlfriends face.

and the parser correctly tagged 'nut' as a VBP (i.e. Verb, non-3rd Person Singular resent) i.e. that 'nut' in this context is a verb, and not a noun.

Here's what the tagging returned (all words with tags)

I/PRP nut/VBP on/IN my/PRP$ girlfriends/JJ face/NN ./.

Tag Descriptions: http://stackoverflow.com/a/1833718/325521

Reply on news.ycombinator.com