Before anyone says that example (or any of the other fluent but _completely nonsensical_ continuations in the article) shows some sort of "understanding," please explain what you would define as understanding?

I would (and I think anyone would) offer an operational definition: there is some class of questions to which this system could reply with sensible, actionable responses. Obviously the present system is not able to "understand" and answer simple arithmetic problems that a first-grader could answer instantly. Given that, would there be any point in expecting it to answer any other logical query that could be of use in one's work? (See the "medical" example in the article, about how to drink hydrochloric acid.)

The only question it appears to answer is, "given some words, what are other words that are likely to follow them in a typical blog post?" The fact that the words are syntactically correct is unimportant, when the fluent words convey no information relevant to the input.