1. Assuming the remote service only sends down streaming audio, this doesn't work for blind people that must use a refreshable Braille display, e.g. deafblind people. Perhaps one could hack a way to get their local screen reader to render specific text on the Braille display, but probably not without that screen reader speaking the same text. That leads me to...
2. A blind user is already running a screen reader, with its own text-to-speech engine, configured the way they want it. Adding a remote screen reader to the mix would mean two different TTS engines, and the user would need to have a way of configuring the remote one, e.g. to adjust its speed. For blind people, TTS settings are very personal.
3. The remote screen reader and the local one may clash on keyboard commands. And, depending on the screen reader, this is another thing that the use may have customized already; for example, some screen readers have desktop and laptop keymaps.
4. Also speaking of keyboard commands, some of them might not be implementable in a browser-based application. It's common, at least on Windows, for screen readers to use non-standard modifier keys, e.g. Insert or Caps Lock.
I hate to say this, but if there was one place I'd look for vulnerabilities within a purportedly-secure environment, screen readers would be near the top of the list.
Then again, I guess my pipedream is equivalent to "re-implement the chrome accessibility system except somehow magically without any user-after-free or etc. bug" :(
So that solves problem #2 in my list (TTS), but may not be an adequate solution for #1 (Braille). And that still leaves the problem of keyboard commands.
However, I think Ins sends a normal keydown/keyup and while capslock is a bit more complicated https://stackoverflow.com/questions/39016292/keydown-event-i... suggests it's not that hard to capture it (for front end javascript values of 'not that hard' ;).
Using the 'correct' keybindings for any given screen reader would presumably require a sort of javascript 'skin' to handle that, of course, and that doesn't solve the key clash problem at all but I'm not sure what can really be done about that except maybe remap things that are in the way on the browser side.
(spitballing in the hopes some of my random thoughts give you ideas ;)
Heh, I get the sarcasm there. I appreciate that most developers don't know anything about this stuff. That's why I freely share what I know as much as I can.
> (spitballing in the hopes some of my random thoughts give you ideas ;)
Thanks for that.
I think the real problem here is the conflict with the local screen reader. That's also a problem for the screen-reader-specific modifiers (Caps Lock and Insert); the local screen reader intercepts those key-down and key-up events, so they will never get through to the browser.
Yeah, this is where I was thinking client-server, wherein the reader would (in pipedream world) send some sort of request through to the other end to move between UI elements ... but (a) that requires said protocol to exist (b) we're back to attack surface problems.
Though ... I wonder if, rather than trying to put together all-of-a-chromium, you could do something with a seriously locked down build based on the tauri project (which I think it was you mentioned upthread somewhere and I'm now going "ooh" at) but I'm aware I'm handwaving a lot there.