350 karma · joined May 7, 2017
https://en.wikipedia.org/wiki/List_of_long-distance_footpath...
https://en.wikipedia.org/wiki/List_of_longest_cross-country_...
https://en.wikipedia.org/wiki/Long-distance_trail
https://en.wikipedia.org/wiki/List_of_long-distance_hiking_t...
https://en.wikipedia.org/wiki/Long-distance_trails_in_the_Un...
https://en.wikipedia.org/wiki/GR_footpath
First, you think you have a neat idea, but minutes later you have a Herculean task at hand ;)
If you misspelled "college" as "collage", or you misspelled "three" as "tree" the word you typed incorrectly happens to be an actual word itself! Correcting these types of errors is called "real word spelling correction".
Real-word spelling errors are words in a text that, although correctly spelled words in the dictionary, are not the words that the writer intended
https://www.researchgate.net/publication/221628953_Real-Word...
And even if there are several approaches evaluated, the problem is far from solved.
But neither spell checking nor its more advanced siblings spelling correction or automatic spelling correction is solved. Here is an example:
"I want to by Apple"
"I want to be Apple"
"I want to buy Apple"
Because "by" is a genuine English word, a spellchecker based on a simple hashtable will not detect the error, which obviously exists. We need to look at the context of the whole sentence or even document. Today we can use deep learning and word embedding to solve/improve that problem.
A simple spelling corrector that is based on edit distance would always prefer "be" over "buy" because "be" is the more frequent word of both.
So the risk of all the easy accessible libraries for a certain task is, that you think the problem is solved, but you overlook many problems because the topic is much more diverse and deep than you would think at the first glance
Another point is that while hardware improved, also the expectations have risen. We expect milliseconds latency instead of seconds, we often require the software to work in a multi-tenant architecture with thousands or millions of concurrent users.
So even for seemingly simple and straightforward tasks, and despite all the exciting advances in hardware, always new challenges present themselves.
30x more queries & docs per dollar spent;
20x speed compared to Lucene
SymSpell can be augmented with a Weighted Edit distance giving a higher priority to pairs that are close to each other on the keyboard layout or which sound similar (e.g. Soundex or other phonetic algorithms which identify different spellings of the same sound). There are two SymSpell implementations with a Weighted Edit distance available: https://github.com/MighTguY/customized-symspell https://github.com/searchhub/preDict
An here a benchmark between Norvig's spelling corrector, BK-tree and SymSpell: https://towardsdatascience.com/symspell-vs-bk-tree-100x-fast...