Hyperlink Maximalism

thesephist.com

9 points·thesephist··3 comments

> When I’m reading something and don’t understand a particular word or want to know more about a quote, when I select it, my computer should search across everything I’ve read and some small high-quality subset of the Web to bring me 5-10 links

IIRC Google Books (used to?) do something similar. Often I'd come across a word or phrase I did not know, I'd highlight it and the dictionary definition would show up in a small modal. I really enjoyed that feature.

That looks like a quite powerful tool, using shared n-grams to find similar concepts in seen documents.

Two problems with it:

* Combinatorics is an unforgiving mistress. This is why we can't even brute force Go/Paduk. A combinatorial explosion of data and links between things. Works great in toy settings, but if you try to scale it up, even just to a single person's experience, it would quickly overwhelm the computing capabilities of the entire world.

* Very brittle compared to natural language, so many things that should be linked are not, even things that seem trivial to humans. If you solve this, you'll have the holy grail of knowledge management. The notorious difficulty of language translation is an example of this. It works great for the first 80% of things, but even improving that by 5% requires herculean effort.

I'm trying to do a very restricted subset of this by parsing natural languages to automatically look up words and group words by their dictionary forms. It feels like I'm already approaching the limits of what is possible, or at least what I'm capable of imagining.

Reply on news.ycombinator.com