Hacker News Reader

Comment by idoubtit | Hacker News Reader

In what sense are these theories a "basis" to NLP? Did they have any influence? Do they bring any practical contributions? I suspect a slight similarity between popular domains (Wittgenstein and NLP) was contrived into an article that seems very light on the W part.

The "Wittgenstein’s theories" that appear here is just that "the meaning of a word is its use in the language". If such a plain concept was all of Wittgenstein’s theories, he would be long forgotten.

For centuries, dictionaries have presented words through one or several explanations as well as quotes and examples. 150 years ago, Émile Littré wrote a wonderful French dictionary that contains 80,000 words and about 300,000 literary quotes. He knew no word has a simple and permanent meaning, and that one needs to know many real world contexts to get a fine view on a word.

View on HN

Yeah, it's a weird kind of philosophy clickbait or something. An only slightly less valid alternate subheading might be "Football is the basis of all modern NLP"[1]

For Wittgenstein - in philosophy, one always refers to the early or later Wittgenstein; they're totally different, and saying 'for Wittgenstein' without specifying which one doesn't make a lot of sense. The early (Tractatus) one had a picture theory of language.[0]

[0] https://en.wikipedia.org/wiki/Picture_theory_of_language

[1] "One day when Wittgenstein was passing a field where a football game was in progress the thought first struck him that in language we play games with words. A central idea of his philosophy, the notion of a ‘language-game’, apparently had its genesis in this incident." - Norman Malcom, Ludwig Wittgenstein: A Memoir

I can't resist a more entertaining extract from that book:

"My wife once gave him some Swiss cheese and rye bread for lunch, which he greatly liked. Thereafter he would more or less insist on eating bread and cheese at all meals, largely ignoring the various dishes that my wife prepared. Wittgenstein declared that it did not much matter to him what he ate, so long as it was always the same. When a dish that looked especially appetizing was brought to the table, I sometimes exclaimed 'Hot Ziggety!' - a slang phrase that I learned as a boy in Kansas. Wittgenstein picked up this expression from me. It was inconceivably droll to hear him exclaim 'Hot Ziggety!' when my wife put the bread and cheese before him. ...

One of Wittgenstein's favourite phrases was the exclamation, 'Leave the bloody thing alone!' He delivered this with a most emphatic intonation and mock solemnity of expression. It had roughly the signification that the thing in question was adequate and one should not try to improve it. He used it on a variety of occasions: one time meaning that the location of his bed was satisfactory and it should not be moved; another time, that the mending that my wife had done on a jacket of his was sufficient and that she should not try to make it better."

That sandwich anecdote reminded me to post a link to this essay about Tractatus as a demonstration of the "autistic cognitive model par excellence."

http://autisticsymphony.com/wittgenstein.html

"Ludwig Wittgenstein was almost certainly autistic. Several notable psychiatrists, such as Christopher Gillberg in A Guide to Asperger Syndrome, have written extensively about the evidence backing this assertion."

Thanks!, that was very interesting. I don't remember reading anywhere that LW had Asperger's (which is what that writer means by 'autistic' I guess), but it seems to fit perfectly. I've just tried to get to know him without labelling. I was close friends with people like Nietzsche and Kierkegaard, but never felt friends with LW hehe. I didn't get far into the Tractatus, probably just a few lines. But I've enjoyed everything else he wrote, always thought-provoking. A fascinating character, with a great sense of humour--he had a friend with whom he used to exchange picture postcards with very silly irrelevant things written on them. Wish I could remember what book I saw those in. Monk, maybe. Also, he was born extremely rich, but believed money corrupts people, so gave it all away to his siblings. When asked why them, since they were already rich, he said 'They're already corrupted'...

(Hmm It just struck me, maybe that explains some of Thoreau too? He didn't seemed incapable of friendship in the normal sense, said No to everything, preferred solitude, seemed utterly unlike other people, his 'duties to himself' overruled all others etc.)

Someone actually published a case report on it in European Child & Adolescent Psychiatry.

You can read it here https://www.researchgate.net/publication/12521251_Did_Ludwig...

A while ago, I commented here[1] to the effect that there's a good fit between the treatment of meaning in Wittgenstein's Philosophical Investigations (PI) and the neural net / connectionist approach, calling it "decentralised, statistical, emergent" in contrast to more cognitivist ideas.

I was challenged to justify my buzzword-laden characterisation, and I reproduce my response here as it seems relevant to your question:

> Right at the start of PI, the "Augustinian picture" of meaning is set out: "words in language name objects - sentences are combinations of such names" a picture where "every word has a meaning. This meaning is correlated with the word. It is the object for which the word stands." And so (later, #81) we "think that if anyone utters a sentence, and means or understands it, he is operating a calculus according to definite rules." This is the view of language which PI aims to - sorry, buzzword incoming - disrupt.

> By contrast, PI puts the case that there is no central unifying model applicable to all instances of language use, rather: "I am saying that these phenomena have no one thing in common which makes us use the same word for all - but that they are related to one another in many different ways, and it is because of this relationship, or these relationships that we call them all language." (PI #65)

> There follows a discussion of vagueness. In the Augustinian picture where the meaning of a word is an object, and a calculus of these objects is performed, it is difficult to avoid the consequence that meanings are exact. PI uses the example of defining the word "game": "How should we explain to someone what a game is? I imagine that we should describe games to him, and we might add: "This and similar things are called 'games'". And do we know any more about it ourselves? [...] But this is not ignorance. We do not know the boundaries because none have been drawn. [...] we can draw a boundary - for a special purpose. Does it take that to make the concept usable? Not at all! [...] One might say that the concept 'game' is a concept with blurred edges."

> So much for "decentralised" and "statistical". As far as "emergent" goes, I think even with the large variance in readings of PI it's uncontroversial to say that it seeks to ground meaning and understanding in relation to "customs" or social practices rather than in some variety of metaphysical correspondence between language and reality required by different variations of the Augustinian picture. In this sense, meaning emerges from the use of words relative to these cultural forms.

> Connectionism, as an investigative paradigm, (oops, buzzword!) is simpler (I believe) in that it doesn't require the identification of an actual realised model (or "mental mechanism") such as a neural encoding of a "language of thought" or cognitive frames, etc., in the brain - it "just" requires that a bunch of simple elements can result in complex rule-following behaviour without needing to explicitly encode the rules. Hopefully the quotes above will go some way to indicate how this programme is philosophically somewhat in tune with PI.

> (Indeed the extensive sections on samples and teaching language games are eerily reminiscent of descriptions of training neural nets, now that I think of it... "How do I explain the meaning of 'regular', 'uniform', 'same' [...] if a person has not yet got the concepts? I shall teach him to use the words by means of examples and by practice - And when I do this I do not communicate less to him than I know myself." (PI #208))

1: https://news.ycombinator.com/item?id=10158214

NLP via word associations might get you closer to capturing meaning in a Wittgensteinian conception than other methods (I happen to think the PI should be required reading for people studying NLP). But he would probably say the inputs into an NLP model would need to capture a much wider range of human culture and behavior beyond co-occurrences and symbols.

And even if you had a very complete model - for him "meaning" something through an utterance was likely something inextricably linked with being human engaged in social acts. (See: the private language argument https://en.wikipedia.org/wiki/Private_language_argument)

One thing I learned from reading Wittgenstein is that language is far more than speech and words. If you want to understand language, or rather "communication" in a Wittgensteinian-way, I think you'd have better luck analysing YouTube videos, where body language, environment, context, intonation, prosody, implicit emotion, etc is far more visible, rather than just analysing text.

It's such a weird, but somewhat freeing perspective on language, where I once thought that the for instance the whistle language is absolutely fascinating and perplexing, it just makes perfect normal sense to me now. As long as you can recognize the human through the medium, there is room for language, and that's so cool! There's so much room for playing with communication and interaction I never thought about thanks to his writing, it also really makes me understand Jazz as a "language" way better, I never understood the almost pretentious way people talked about Jazz, "it's different, hard to explain if you never experienced it for yourself!", it really, truly is a conversation in music, super interesting.

> he would probably say the inputs into an NLP model would need to capture a much wider range of human culture and behavior

I get what you're saying - that something deserving the name "language" way outstrips the simple word embedding strategies in TFA, right? and I wholeheartedly agree - but my point is that the position of PI, at least as I read it, is fundamentally opposed to the idea that modeling - what he calls "a mental mechanism" - is how language proceeds. This is one of the major points of divergence between PI and the Tractatus (which posits an explicit correspondence or modeling relation between language and the world.)

The most explicit formulation of this that comes to mind is in PI #689:

> "There must surely be a further, different connexion between my talk and N, for otherwise I should still not have meant HIM."

> Certainly such a connexion exists. Only not as you imagine it: namely by means of a mental mechanism.

The point is very much that PI's emergent view of meaning is quite in tune with the connectionist sentiment that a bunch of simple elements (i.e. neurons, either in the brain or in some AI project) can exhibit the rule learning and rule following capacities at the centre of PI's treatment of language, without any model of the elements of language needing to be explicitly encoded.

The degree to which such a model may or may not exist, embedded within the parameterisation of a language-capable set of neural components, is an interesting question. But even granting its "existence" I think the point would be that the basic properties of the set of components, coupled with the training and samples provide the capability and the "model," if we want to grant its existence, is effectively a by-product - epiphenomenal if you like. :)

There’s a difference between a common sense understanding of a notion and a detailed and organized examination of it.

People had a common sense understanding of gravity before newton and Galileo, but that doesn’t mean they added nothing to our understanding of it.

You’re being refuctive in your framing of his early work. At the same time as Wittgenstein was playing with words, the logical positivists were attempting to go the opposite route, and find language that both expressed meaningful things about the world and was also rigorously well formed; the meaning is a mathematical logical implication formed from empirical observations.

Naturally, they failed. Wittgenstein represents the other end of the spectrum: not only will words not save us from meaninglessness and irrationality, words can lead us straight towards them in the guise of reason.

As always, the celebrated hero is standing on giants, and they symbolize the progress rather than singlehanded executing a large leap themselves. You could apply a similar disdain to most scientists and philosophers we learn about.

Reply on news.ycombinator.com