How do machines learn meaning? | Hacker News Reader

How do machines learn meaning?

blog.lateral.io

This isn't learning by any means - this is just similarity. Machines don't understand what the words mean, nor the nuance of how to use them via this method.

You are going to be downvoted into oblivion by the AI zealots here at HN. But, of course, you are right!

You say this as though you have a definition of what it means for something to "understand" words. It's easy to make a claim, why don't you try to back it up?

usually, we assume that a meat grinder does not "understand" meat, that a rice cooker does not "understand" food, and that a telephone or a CD player does not "understand" human language.

Wiktionary terms "understand" as (i) be aware of the meaning of something (so we've just reduced it to the meaning of awareness and meaning), or (ii) to impute meaning that is not explicitly stated.

To put it in more colorful terms, a blind person can understand the difference between green, yellow and brown bananas, but they usually cannot understand the visual aspect of color.

By the same rationale, a vector does not understand the word it describes any more than the telephone book understands the people listed in it or the city that they live in.

We are not talking about just a vector, but rather vectors trained in a specific way along with cosine similarity. You could easily argue that this system can tell you that one words has a different meaning from another word, or that two words have similar meaning. Further, it learned this without these relationships being explicitly stated.

Once you start to try to define aspects of cognition explicitly, things very quickly get ambiguous. Also, these conversations usually go along the lines of:

1. State a definition.

2. See that computer matches.

3. Decide it's wrong after all and try to change it so that computer can't match it.

4. Repeat until we find a definition that excludes computer.

I think it's a fascinating topic, but the above pattern is fairly disappointing.

Well, if you use the term "machine learning", you've got to accept that the "learning" there doesn't have all the same connotations than the word "learning" has in general. But if we just define learning to be getting better at something "autonomously", machine learning has that property. The programmer codes the learning algorithm, but the program gets better at the task itself by example.

> this is just similarity

That may be all that "learning" and "understanding" are.

I'm not ML expert (especially in the natural language space), but I can't help thinking something is missing here. I know a ship 'is-a' boat. Cats and dogs are unrelated animals etc. Maybe the colocated words are a very good approximation of a dictionary or real meaning?

well the idea is that a word's meaning can be derived from its context (those context clues from grade school strike again). The co located words are simply the context. Since "ship" and "boat" frequently occur in similar contexts, they are similar words.

Honestly, this blog post felt like they were trying to explain word2vec for the hundredth time, but tried real hard to explain it differently--so much so that it just made it more confusing.

The author is refreshingly straightforward about what they are doing: "The key insight, made at the beginning of the information age, is to replace word meaning with something that machines can actually measure."

Of course, if you do too much of this sort of meddling with usage, it will really mess with the machines' method for finding 'meaning' in the language. I expect humans would cope with such meddling much better, because they have the advantage of natural understanding.

So this is my research area.

Distributional vectors are a proxy to word meaning. Words with similar vectors have similar meaning or are semantically related in some way. But usually, you just measure similarity by a single number: the cosine similarity between the two words' vectors.

This number can tell you words are related or not, but it can't tell you how they're related [1]. There's been a good deal of work in automatically identifying that "ship is-a boat" (which is called hypernymy) or cats and dogs are unrelated animals (cohyponomy), but it's still being perfected.

But it is useful. Words that are similar in meaning can be treated similarly. As a bad example: maybe I know that "anger" has negative sentiment, but I don't know what sentiment "furious" has, but I can infer it probably has negative sentiment since it's so similar to "anger".

[1] There's a good deal of evidence that words that have high cosine similarity are more likely to be cohyponyms.

Reply on news.ycombinator.com