Hacker News Reader

Comment by canjobear | Hacker News Reader

It seems like there's a hard limit on how well this approach can work, defined by the extent to which the distribution of words in texts uniquely identifies their meaning without reference to any external semantics.

With monolingual corpora and no semantic grounding, all you can say is that certain words (or letters, morphemes, or whatever) covary in certain ways that are similar to the ways in which other words covary in the other language. It's super interesting that this is enough to identify some translation pairs across languages.

View on HN

In particular I wonder if it can get "left" and "right" the correct way around.

Excellent example!

I'd imagine you can potentially get that by connections with strength, weakness, dexterity, etc.

I'm not sure I understand you... But the parent's comment is quite interesting, how could the system end up with a correct left/right understanding or our world, instead of a mirror world? Without experiencing the outside world, it seems impossible!

The remark is that the two words, while apparently interchangeable, in actual use are not. One is often used with positive meaning related to dexterity and so on. Also, they have different political meanings. So they are not at all symmetric in actual use, much like every other word pair

Not all of those differences are cross-cultural.

This is a good point. Still, for instance, the binomial left/right to indicate political position is common across many languages

They definitely work in a lot of languages.

https://en.wiktionary.org/wiki/derecho#Spanish https://en.wiktionary.org/wiki/direito https://en.wiktionary.org/wiki/diritto https://en.wiktionary.org/wiki/sinister#Latin https://en.wiktionary.org/wiki/dexter#Latin https://en.wiktionary.org/wiki/laevus#Latin https://en.wiktionary.org/wiki/%CE%BB%CE%B1%CE%B9%CF%8C%CF%8... https://en.wiktionary.org/wiki/recht#German

and many languages related to these. Some Slavic languages have a close cognate between words meaning 'right (direction)' and words meaning 'right (correct)', although I didn't immediately find examples in current use on Wiktionary where the two forms are identical. A "dated" example is Polish

https://en.wiktionary.org/wiki/prawy#Polish

Edit: a Finnish friend pointed out that this works in Finnish, which is European but not related to any of the other languages above.

https://en.wiktionary.org/wiki/oikea#Finnish

I don't know if these linguistic phenomena definitely occur in non-European languages, but I think that many people in the Middle East, many parts of Africa, and South Asia consider the left hand unlucky or impure not only because it's weaker in most people but also because it's conventionally used when going to the toilet. It would be a little surprising to me if that attitude didn't show up in languages from those places too.

Edit: it looks like it works in Arabic too

https://en.wiktionary.org/wiki/%D8%B4%D9%85%D8%A7%D9%84#Arab... https://en.wiktionary.org/wiki/%D8%A3%D9%8A%D8%B3%D8%B1#Arab... https://en.wiktionary.org/wiki/%D8%A3%D9%8A%D9%85%D9%86#Arab...

I guess there might be the occasional reference to the heart being on the left, or to most people being right handed. So among human languages it would eventually learn. But it wouldn't work as Star Trek's "universal translator": aliens wouldn't share these traits with us. I suppose we could get lucky and find an alien textbook describing CP violations.

Very interesting. Can a language define all it's concepts with language alone? I.e

Can a language map all it's concepts to words and other concepts defined by just words.

Is there some sort of Godel's incompleteness theorem for language?

> Can a language map all it's concepts to words and other concepts defined by just words.

I don't see why natural language would have this property. It is certainly possible to construct codes such that the distribution over codewords does not have this property.

I also don't see why it would, but then that would of course be a limit to the idea in the paper.

I'm amazed that what the paper shows even works.

It seems extremely unlikely to me that it wouldn't. I conjecture that no such word exists in any widely used human language. This would be a word who's meaning is not contained in the span of meanings of all other words in the language.

Can you think of one in english (or your native language)? You won't find it in any dictionary, since all those words have descriptions in terms of other words.

Gullible?

easily deceived

The example I've been thinking about was 'pain', which, according to Wikipedia, is "an unpleasant sensory and emotional experience associated with actual or potential tissue damage." The native Dutch word "Gezelligheid" has no direct English translation and thus apparently has a wiki page in English [0]. Now I wonder what the relevance is of the distance between word vectors from different languages relating to nearly the same concept (e.g Gezelligheid to Cozyness)

[0]: https://en.wikipedia.org/wiki/Gezelligheid

actually, it's theoretically a hard problem -- https://aclanthology.info/pdf/J/J86/J86-4003.pdf

I would have thought that they needed to 'ground' (https://en.wikipedia.org/wiki/Symbol_grounding_problem) at least one shared reference word/concept so they could proceed to define everything relative to that and thereby communicate effectively and know how to map different languages to the same latent space. But I guess perhaps one can infer/approximate it statistically -- https://cis.upenn.edu/~ccb/publications/discriminative-bilin...

Reply on news.ycombinator.com