Worse still, while the Gio layout engine is pretty good at culling non-visible stuff, ultimately the framework doesn't know what will be visible until we're in GPU land, so there would have to be a ton of bookkeeping added to test if, for example, some text was clipped or if things were overlapping.
In practice, the best way to model things would be to just allow the developer to do something equivalent to pushing fields full of ARIA information like in the browser. But now you're losing all the advantages of the immediate mode and layout engine, etc.
It's a lose/lose situation, unfortunately.
I think the real solution is somewhere in the middle. I think screen readers need to grow some OCR abilities, and libraries like Gio need to learn some better navigation tech, like supporting tab, keyboard nav, etc. Something like that.