Hacker News Reader

Comment by neves | Hacker News Reader

Newbie here. Can someone explain what would I use these vectors for?

Besides what others have already said: word vectors are usually calculated from the context they appear in, the other words surrounding them. The vector representation can be intuitively understood as representation of the context the word appears in. The number of words we take into account for that context is called 'window'.

This has very interesting possibilities: you can complete sentences where a word is missing (you have a context, so you can search for the best matching word vector), use it in text autocorrection tasks, and other classic natural language processing problems.

Learned word representations are also coherent between them, so you can use them to make analogies (the distance from 'Spain' to 'Madrid' is similar to the distance from 'France' to 'Paris'), so they implicitly hold some of the semantic info between words.

It can also be used to find related words. Synonyms, antonyms and related words have similar representations. For example, 'facebook', 'twitter' and 'instagram' have similar vectors (vectors with similar directions). But you can also try with famous musician or band names, tech related terms, etc.

Finally, word vectors, unlike other language models, can store representations for large windows and vocabulary sizes in a few GB, which is another useful property in certain situations, and makes them easy to handle.

At one level, it’s simply a vector of weights. In a simple 1-of-N (or ‘one-hot’) encoding every element in the vector is associated with a word in the vocabulary. The encoding of a given word is simply the vector in which the corresponding element is set to one, and all other elements are zero. It's all about Natural Language Processing.

If you are interested in more, check out these excellent reviews by Adrian Colyer posted in The Morning Paper.

https://blog.acolyer.org/2016/04/21/the-amazing-power-of-wor...

that's a nice link but the excerpt you quote is kind of misleading; the word vectors in this case are not one-hot encodings. They are learned, continuous representations. But one-hot representations are also a kind of word vector.

word vectors are vector representations of each word in the vocabulary. Here they are learned by a neural net. the length of the vector is the # of features. Just for intuition, one feature of a word the NN could learn is the gender of a word, and so on.

Yes, quite correct! The vectors are dense representations.

But the features aren't individually interpretable, in practice. For instance, the 'gender' of a word may have it's signal scattered over several features/dimensions of the learned vector.

To help with intuition, here are a few example word vectors that we might encounter:

"fruit": {food: 0.99, gender: -0.05, size: 0.2}

"king": {food: -0.9, gender: 0.92, size: 0.56}

Building off of what v1n337 stated, though, axis can easily be skewed and rotated such that they're still interpretable, just not obviously so.

Reply on news.ycombinator.com