Hacker News Reader

> In the same way a browser can give the AI the accessibility DOM, Apple could give AIs an easier representation to read and manipulate.

Apps already have such an accessibility tree; it's used for VoiceOver and you can use it to write UI unit tests. (If you haven't tested your own app with VoiceOver, you should.)

View on HN

I have used it actually! It's been years so the fact that it's an accessibility tree just like in a browser didn't come to mind immediately. Both Mac and Windows have such representations for native apps. The actual functionality apps and the accessibility clients support is something like a two-way negotiation. A lot of stuff that should be supported in apps, in theory, is not, just because no client supports it, etc.

Reply on news.ycombinator.com