Hacker News Reader

Comment by PaulHoule | Hacker News Reader

There's no doubt that Noam Chomsky founded a paradigm of academic activity. Linguists can generate an unlimited number of papers and monographs by finding problems and proposing intellectually convincing solutions.

From an engineering standpoint, however, Chomsky's view of grammar has been remarkably barren when it comes to machine processing of natural language. It's made a major contribution to artificial languages but despite a lot of effort it hasn't added much performance to what can be done with statistical methods.

I'd agree that a hidden Markov model that does POS tagging with high accuracy doesn't provide an intellectually satisfying model for "how language works", but you don't need to have a model for "how language works" in order to use it.

View on HN

I feel there is excessive emphasis on "what Chomsky said" and "what Chomsky did". Norvig chooses to point out that the principles and parameters framework is, let's say, imperfect, but, well, Chomsky would agree. Moreover, if you zoom out and stop obsessing about quotes from "Syntactic Structures", you will realize that a lot of the work that's being done in theoretical linguistics is not quite as barren. Yes, statistical methods for (say) anaphora resolution can be extremely efficient, but basically very few people had thought about anaphoric relations in any systematic way before generative linguistics came around.

Moreover, rule-based NLP approaches also have their place, and they are often the direct result of theoretical advances. A case in point is the modelling of morphophonology (which is necessary for spell checking, dictionaries and text generation for morphologically complex languages): many successful approaches are those based on finite-state machines, which could not have happened without Johnson and later Koskenniemi using them to formalize the rule-based approach pioneered by Halle and (yes) Chomsky (well, not quite, but this is still the point of reference for rule-based phonology).

(I am a theoretical phonologist, but my colleagues who do actual NLP work of this type tell me that statistical methods aren't that great for the sort of work they do.)

I absolutely agree. I don't care if the computer that automatically transcribes my voicemail understands speech either - I just want it to be correct.

That technology is impressive to the utmost degree, I just don't think it should be the endgoal of modern science.

There is no guarantee that the Kolmogorov complexity of any interesting system will fit inside the rational parts of our heads. There pretty much is a guarantee that we will not be able to fully understand our own brains, using our brains; the part of our brain that can understand things is just dwarfed by the size of the rest of it. (We really do quite a lot with not very many free neurons.) Even if there is a generative theory that can explain human speech in less bits than a direct lookup table, there's no guarantee we can find it, and the null hypothesis must be that we won't because there is no such theory.

We should look for it, but we should not expect to find it.

I'd agree that an adhoc equation that fits observed core sample data doesn't provide an intellectually satisfying model for "how sedimentation works", but you don't need to have a model for "how sedimentation works" in order to use it.

That's actually from something I worked on a long time ago. People actually use adhoc models of sedimentation and the formation of sedimentary rock for practical purposes. I also think most people suspect that we'd learn something valuable by figuring out the underlying reason why the data fits the particular description.

Reply on news.ycombinator.com