Hacker News Reader

Comment by KingMob | Hacker News Reader

Spell check? Isn't that a well-solved problem at this point?

No. Spell check frequently still gets things wrong if the word is spelled correctly and the sentence is grammatically correct but the wrong word was used.

Can you give me an example? Spell check only checks if a word is in dictionary. It doesn’t check grammar or context.

"Bob went to Venice to pick up the doge."

Where doge is both the name of a title (like duke) but it is misspelt "dog". The use of "Venice" where doge's are could increase a the likelihood of a smarter spell check keeping doge and not correcting to dog. Looking at a wider context might see that Bob is talking about a pupper.

A simpler example would be "spell cheque"

A spelling error, using one dictionary definition, is "an error in the conventionally accepted form of spelling a word" --- mistaking one word for another does not fall under this definition. It is true that we now expect spell checkers to do grammatical checking as well, but a pure spell checker can indeed rely on a wordlist for English (this wouldn't work in languages with more developed morphology and/or frequent compounding).

Ok, but this is a technicality. Spell-checkers have slowly evolved into grammar checkers and what people really want is error correction. Whether people call it a spell checker a minor language issue (and the kind of things humans do all the time).

When teaching for your dictionary, ask: "is it obvious what they mean if I'm not being pedantic?"

Agreed. "Dessert" vs "desert" - mistaking these two is often not a grammatical error (they're both nouns), but is a spelling error (they have quite different meanings, and the person who wrote the word simply spelled it wrongly).

I agree, but this is definitely the kind of spelling error (along with complementary/complimentary, discrete/discreet, etc.) that we normally don't expect our spellcheckers to catch.

We expect different outputs in these two cases, though. A wrong word choice is usually accompanied by a hint that another word may have been intended, while a wrong spelling can be unambiguously marked as a mistake. These two behaviours can be turned on and off independently, and they need two different labels.

I don’t think I agree with your interpretation of the definition.

If I spell the word “pale” as “pal”, that is not an acceptable spelling for the word “pale”, even if it is coincidentally the acceptable spelling for an entirely different word.

If I asked a human editor to spellcheck the sentence: “His mouth dropped and he turned pal.”, the editor would correctly indicate I had misspelled the word.

Spellcheck hasn’t done this in the past because it can be quite difficult. But that’s a limitation of computer capability, not functionality bounded by the definition of the term “spellcheck”.

Cue people mistaking cue for queue

Butt wouldn't you liked if a spell cheque could of fixed these command?

Hah! Apple caught "of" and suggested "consider have instead", but left the rest untouched. Great qed for spell checkers.

Chatgpt fixed it though: "But wouldn't you like it if a spell check could have fixed these commands?"

Finnish would like a word. Take a random noun like kauppa "shop". It has at least 6000 forms: https://flammie.github.io/omorfi/genkau3.html and that's excluding compounds (written as one word in Finnish) like "bookshop" or "shop-manager" etc. etc. And then you have loan words and slang, derivations into other words classes; all of this is impossible to compactly represent in a full-form word list.

Now consider the many other languages of that family ( https://en.wikipedia.org/wiki/Uralic_languages ) – they also have this extreme potential for inflections, but next to no online resources to train language models on or even scrape decent wordlists from.

"would like a word". I see what you did there...

Finnish is very different from most other languages, and does not have the user base to be well represented in training data, but that webpage is ridiculous and does not reflect the actual language. No one in the history of Finnish has ever spoken most of those forms. Grammar describes language, it does not define it!

That's exactly what they're saying. If you write “the work required deep incite”, a traditional spell checker won't catch the mistake (but people consider it a spelling error).

It might sound unbelievable but if you write in multiple languages and mix languages in the same message or sentence, often spell check doesn't work properly. Which is only normal.

I regularly send messages in 4 different languages (living in a bilingual city + frequent use of English and lots of Spanish friends). Sometimes even using 3 languages in one sentence.

Whatsapp kind of improved it now in that you can "activate" two languages at the same time. Apart from that I'm not sure there's much else that can be done.

It's not even that much of an edge case. Brussels is the one of the most international cities in the world, street names exist in 2 languages, a lot of slang and expressions get borrowed from other languages.

Its knot.

How does grammarly exist then? Must be some secret sauce in there.

Solved how? Language is always evolving

Google Docs spellcheck has been really good for few years even before LLMs

Not for German, surprisingly.

LLMs aren't very good in non-English anyway, one thing it does is translate the in- and output to and from English because it has more available information in English.

(disclaimer: single data point, a lot of assumptions in the above)

Reply on news.ycombinator.com