Do Colorless Ideas Sleep Furiously? (1997)
mit.edu
mit.edu
> you were asked to compose not more than 100 words of prose, or 14 lines of verse, in which a sentence described as grammatically acceptable but without meaning did, in the event, become meaningful. The sentence, devised by Noam Chomsky, was: colourless green ideas sleep furiously.
Four winning entries here: http://archives.conlang.info/ga/farzhi/shiarweilwoen.html
I'm not so sure. GPT-2 says
log P("Colorless green thoughts sleep furiously.") = -53.64797019958496
log P("Furiously sleep thoughts green colorless.") = -65.46656107902527
The ungrammatical one is lower probability. But those are famous sentences, and probably present in the training data, so let's try
log P("Colorless blue ideas hibernate angrily.") = -60.12953460030258
log P("Angrily hibernate ideas blue colorless.") = -70.02637100033462
Yes, Chomsky mentions this in a footnote. But as far as I know, it hasn't been tried with modern language models.
There's been some interesting work that tries to reproduce grammaticality judgments in terms of language model probability after controlling for length and lexical content. It turns out it works pretty well. For instance https://arxiv.org/pdf/1910.14659.pdf
> "... a structural analysis cannot be understood as a schematic summary developed by sharpening the blurred edges in the full statistical picture. If we rank the sequences of a given length in order of statistical approximation to English, we will find both grammatical and ungrammatical sequences scattered throughout the list; there appears to be no particular relation between order of approximation and grammaticalness. Despite the undeniable interest and importance of semantic and statistical studies of language, they appear to have no direct relevance to the problem of determining or characterizing the set of grammatical utterances. I think that we are forced to conclude that grammar is autonomous and independent of meaning, and that probabilistic models give no particular insight into some of the basic problems of syntactic structure."
I do think it's an important point for people to recognize. Scientific theories don't arise on their own out of large-scale statistical analyses. There is a lot of faith being put in deep learning methods these days, which are great for prediction, but not inference.
> If we rank the sequences of a given length in order of statistical approximation to English, we will find both grammatical and ungrammatical sequences scattered throughout the list; there appears to be no particular relation between order of approximation and grammaticalness.
It's totally not clear that this would be true with modern language models, after you control for (1) the length of the sentence and (2) the words in the sentence (as mentioned in the thing I linked above).
Having said that; and having formally studied linguistics at an undergrad level, "Furiously sleep ideas green colorless" is different than "Colorless green ideas sleep furiously" in that the former is not grammatical.
It is also the case that non-grammatical sentences can have meaning. For example, I believe that most English speaker would agree that the sentence "me happy" is not grammatical, and indicates that the speaker is happy.
I never get tired of the internal contradictions people rely on to dismiss what is in front of their face. The start of this article declares that the phrase is meaningless and then ends with a fake revelation that the author has defined it's meaning. The author is so determined that the initial reaction is meaningless that we need to read three paragraphs about palming off nature to a language game.
Bring back Steve Jobs.
Racter would definitely say so.