Worse still, while the Gio layout engine is pretty good at culling non-visible stuff, ultimately the framework doesn't know what will be visible until we're in GPU land, so there would have to be a ton of bookkeeping added to test if, for example, some text was clipped or if things were overlapping.
In practice, the best way to model things would be to just allow the developer to do something equivalent to pushing fields full of ARIA information like in the browser. But now you're losing all the advantages of the immediate mode and layout engine, etc.
It's a lose/lose situation, unfortunately.
I think the real solution is somewhere in the middle. I think screen readers need to grow some OCR abilities, and libraries like Gio need to learn some better navigation tech, like supporting tab, keyboard nav, etc. Something like that.
Just IMO, it's unrealistic to expect to have an accessible GUI that stores no state, does none of that type of bookkeeping, has no OOP model, and only outputs to pixels. The point with these type of assistive technologies is that the user can't work with that type of visual data, they need more state presented to them. Sure, it makes everything easier to develop when you cut it all out and only use immediate mode, but that's exactly the problem: everything else has been cut out, on purpose.
https://machinelearning.apple.com/research/creating-accessib...
Forgive my ignorance, but what should happen here? Let say I have a scrollable text pane with an entire novel in it.
I agree there should still be a mechanism to pass more accessibility data than OCR could extract. What is the bare minimum information that is required? Pretend there is no UX model -- arbitrary things could be presented (like in a video game).
In particular, with Gio there isn't necessarily a single set of widgets or UX. Gio has a small set of Material Design compliant widgets, but is mostly a library for composing immediate mode graphics. For example I have several custom widgets, some that only interact with the keyboard, some that don't have any text at all (just graphics or animations), one is a akin to a 2D-scrollable map, click-to-drag Google maps style. I'm not really sure where I'd begin in making these accessible. How should something like Google Earth be made accessible, ideally?