Hacker News Reader

Comment by a-priori | Hacker News Reader

Before you start picking up a library like this, keep in mind that no immediate-mode GUI library I'm aware of (including this one from what I can see in the source code) has support for assistive technologies such as screen readers.

To them the whole UI is one big black box, so your program is completely useless to anyone with a wide range of disabilities.

View on HN

The immediate mode Gio (gioui.org) toolkit has support for accessibility (on Android for now), IME and, recently, RTL text layout.

Disclaimer: I'm the author.

That’s great, thanks for the letting me know!

I can also report some modest progress on my own work on accessibility of immediate-mode GUIs. I have a branch of the Rust egui library [1] that has basic accessibility on Windows using my AccessKit project [2]. I do have a long way to go to make this fully usable and ready to submit upstream, especially when taking non-Windows platforms into account.

[1]: https://github.com/mwcampbell/egui/tree/accesskit

[2]: https://github.com/AccessKit/accesskit

pretty awesome lib!

Do operation systems have APIs, so the program can send them what is on the screen? I mean just description like: There is the button with title "Push me" in the rectangle(50, 50, 100, 70), etc.

Or are there any "screen reader" libraries?

There's a10n standards defined by w3c, https://www.w3.org/TR/using-aria/.

Yes. I don’t know about Windows but I know that for OSX you need to implement the NSAccessibility protocol in AppKit. This gives you a way to describe the hierarchy of the UI and to notify the system of important changes.

Yes they do, at least Apple, Google and Microsoft ones.

On GNU/Linux, I think the only thing that exists in the Gtk+ features that were originally sponsored by Sun.

Even systems like Qt, where things are drawn using non-native widgets support that (I was long proponent of wxWidgets, because I thought assistive tech is only support only if you use native widgets. The idea of API where you explain what you are showing, and how to interact with it never came to my mind, but with ARIA - like any browser that's the way to go).

Not only that, but I thhink assistive tech should allow you do to easier GUI testing too, but haven't digged into that idea yet (probably folks already are)

Yes, OSes have APIs. Another commenter talked about the Mac one, Linux has AT-SPI and Windows has MSAA (historic) and UIA. If I'm not mistaken, they generally are about representing the UI as a tree of items as this is what the accessibility hardware expects / works best with.

Aren't you then trying to bridge an immediate mode API (rendering controls) and a retained mode API (assistive tech).

Yes. This has influenced the design of my AccessKit [1] project. With AccessKit, the application (or GUI toolkit) pushes tree updates to the platform adapter, which maintains a complete tree that it uses to implement the platform-specific accessibility API. Each application-supplied tree update can be a full tree or just the nodes that have changed. So an immediate-mode GUI can push a full tree every frame. There just has to be some way of keeping node IDs stable across frames.

[1]: https://github.com/AccessKit/accesskit

I've been using an immediate mode api that generates html elements in web, so that you do get the things that html offers. I basically have a thin wrapper around http://google.github.io/incremental-dom/ (I'm using it from wasm) -- calling it per frame has actually been reasonable perf-wise.

Thanks for making the post I was coming to make :)

While it's popular to mock electron, it does make it easy to make accessible cross-platform apps, whereas most of the "minimal alternatives" are entirely inaccessible.

I think egui (Rust) supports screenreaders

Seeing that it uses char* for strings, I'd also assume it doesn't support unicode and thus non-Latin scripts.

Raygui supports unicode for input and output.

https://github.com/raysan5/raygui/issues/99

Reply on news.ycombinator.com