Hacker News Reader

Comment by flohofwoe | Hacker News Reader

...and yet the first thing I switch off in every tool is the spellchecker, because they get utterly confused with the mix of English, German and my local dialect that I'm using when communicating with different people. I'd say doing a spellchecker "right" is apparently still a major feat of software engineering ;)

View on HN

I wish it just struggled with the mix of multiple languages, it can't even handle my native one. A plural compound word in translative case with a clitic? I can only dream about it, it can't even do all the cases reliably.

What language is that?

I guess what I'm wondering is how obscure it has to get (or perhaps how low the GDP of the people speaking a language has to be) before custom spell checking rules aren't considered worth it to bigcorps anymore. Though I'm also kinda interested in what this weird language thing is you're talking about.

Based on the features; Hungarian, Finnish, or Estonian.

Microsoft has an okay Finnish spellchecker. It's really a shame no Finnish university has developed an acceptable open source Finnish spellchecker though AFAIK.

Word-list based checkers actually work for English, which gives an impression that they would work for all types of languages. For agglutinative languages (which I think is about 50% of world languages) not so much, since the list of valid words is basically infinite.

But combined with a high-quality root word list it's possible to write a computer program that returns if a given input word is a valid form of some known root word, i.e. a spellchecker.

In English, you can think the problem same as writing a spellchecker that can tell if an input word is a valid chemical name for a molecule, like 1,3,7-trimethylxanthine (caffeine) or adenosine-5'-triphosphate (ATP). Clearly, we can not write an exhaustive list of valid words here. But it's pretty easy to write a computer program that can tell if the input word at least seems like a valid chemical name devoid of typos.

Could be Italian, where there are words like `portatemelo` ~ `you bring that to me` that google routinely gets wrong.

I often write in a mixture of Scots and English, and spellcheckers are nae guid for that use case. The problem is solved, however, on iOS and Android thanks to SwiftKey[1].

[1] https://dsl.ac.uk/our-publications/scots-predictive-keyboard...

+1 to SwiftKey. I use a mix of English and Estonian and SwiftKey handles it amazingly. Almost never suggests the wrong language when I start typing.

Honestly I can't use the default keyboard in iOS anymore because it lacks multilingual support. How it still isn't a thing in 2020 boggles me.

Yeah, to add to it a lot of Germans will use American English but some British English and my spellchecker can't handle the mixture. Off topic but despide how difficult German has been to learn I've found spelling in German to be far more predictable and straightforward than my native language English.

This is at least partly because all of the German speaking countries got together in 1996 to reform the spelling of the language.

See here: https://en.wikipedia.org/wiki/German_orthography_reform_of_1...

Right, I vaguely remember hearing about this. Another benefit is you usually have a fairly good idea how a word is pronounced, which is not the case for English or French in my exprience. In some ways I guess American English attempts to standardise and fix some rough edges of English.

It tried, but I'm not sure how much it succeeded.

American English changes a few cases, but there are often exceptions (color, favor, but four, glamour; -ize, but still advertise, compromise etc).

There are too many differences for me to summarize (American and Oxford English spelling there), but I think the problems with English spelling are much deeper than American English can fix.

https://en.wikipedia.org/wiki/American_and_British_English_s...

Reply on news.ycombinator.com