Comment by lewisjoe | Hacker News Reader

Comment by lewisjoe | Hacker News Reader

Parent Full thread

I'm part of the team that makes Zoho Writer's grammar checker[1] and we did go with a modern neural-net based NLP technique to detect grammatical errors. It had a huge win in terms of the number of errors it caught, but we could never tell the user what exactly is wrong with that sentence, because.. well, deep learning.

So we've taken a hybrid approach where a modern AI flags errors and then run it through a rule engine to try and figure out what exactly is wrong. This way we get the best of both worlds. That lack of explanation is also why Grammarly is sticking with GOFAI systems perhaps. But the future of this problem area is certainly AI augmented with rule engines.

[1] - https://www.zoho.com/writer/free-grammar-checker.html

Yes, that's currently the problem. I'll be that Grammarly is doing something similar itself.

Do you support Japanese or other East Asian languages?

But if the NLP is able to flag more errors than the rules engine, how could the rules engine explain the errors?

I'm not sure I understand what you're asking, but neural net-based NLP can't explain itself while the rules engines just use simple pattern matching techniques, so it can.

OP is saying that the rules engine can't catch a bunch of errors that the ML based NLP can. In other words the ML system has a bigger input domain than the rules engine. I am confused about how they plan to use the rules engine to explain errors that it won't be able to catch at all.

Oh, I expect that you'd use NN NLP to do accurate parsing/tagging/NER then write rules based on the output of that parsing.

If you look at https://explosion.ai/demos/displacy, you can see the high-level information you can extract from sentences nowadays, which better matches our understanding of how grammar rules work.

And that's an approach that does generalize to languages like Japanese.

Boy, I would love to see an API for the analytical tools.

Reply on news.ycombinator.com