Hacker News Reader

Have you considered using ML/OCR to figure out the position of the text relative to the screen? Seems much simpler than relying on accessibility APIs

Thank you for your hard work!

View on HN

I have plans to use ML/OCR to augment results down the road but the AX APIs and ecosystem on most apps (that I encounter, at least) are generally decent. Also, OCR means it won’t understand buttons with just icons, whereas AX APIs can grab em just fine.

Thanks! It’s easily my longest running project at a decade

Reply on news.ycombinator.com