Comment by ClawsOnPaws | Hacker News Reader

Comment by ClawsOnPaws | Hacker News Reader

Parent Full thread

Something that Chromium apps do give you however, for free for the most part, is accessibility. I just tried the GUI version of this client and was not surprised to find out that I could not use it. The new Spotify UI released a few months ago is the most accessible Spotify has ever been. Landmarks, clear labels, headings, and even aria-trickery to automatically announce things using my screen reader. I remember being very frustrated with the old UI's to the point where I chose another service just because it was more accessible, even if it didn't have a desktop app. YouTube Music and Deezer had much better UI's from the get go. At this point, I'm almost happy to see an Electron app. It doesn't guarantee accessibility, but the likelyhood is so, so, so much higher than any modern cross-platform UI framework. I'd almost go as far as to not call these UI's native. Because if they were, if they used native controls, the accessibility would be there. The OS vendors spend a lot of time to make them usable and consistent. Sadly, these UI frameworks don't, or can't. Sure, psst has a CLI, but I only get panics. I can't do -h to find out what I can do with it, I can only call it with a spotify URL and get it to play and exit once it's done. It feels like the cli was included as a sort of testing tool to check the underlying libs and code, and not as a usable version of the app itself - but it's still very early in development so the GUI might be the same. I can't tell.

Here's how these things usually go: first, you start small. You build something you can show other people. You try to generate some traction and build interest around your idea, validating it at a very small scale (friends, family, "Show HN", etc). You gradually expand that cycle of people. You get into a routine of iterating on improvements, adding new features, and gradually scaling things up. You probably get a couple of people to join you on that journey, because the more popular your product becomes - the more asks you'll be getting, and the more work it takes to scale it out. Rinse and repeat.

The reality is that we can't expect every single hobby project to support every single use-case from day one. Painfully for all - the current state of things in this space is such that building accessibility into your product is not trivial and hence it gets postponed "for later".

Now, on to a completely honest question: why don't we have screen readers that can consistently translate the "visual" version to a version people with visibility impairments can understand in a more generic manner, without requiring every single piece of software to adjust itself to every single flavor of a screen reader? I'm honestly asking why can't we solve the whole problem by offloading it to the screen reader itself. Can we solve this by simply offering a text-only command-line like version of every product? (ie. as opposed to building "beautiful"/"designed" experiences that get downsampled by the screen-reader anyways)? Sort of like building a sitemap.xml file that lets the user with a screen reader do everything a user with the full-blown GUI can. Sounds like an opportunity to create a cross-industry standard?

Edit: getting downvoted for comments like this is why I'm honestly considering just closing my account and not participating in any more such conversations going forward.

I upvoted you to combat your downvote because your initial paragraphs was a very reasonable description of the journey of a lot of hobby projects and its unreasonable to expect alpha apps (built using a GUI FX in active development) racing to get out an MVP is going to have accessibility nailed out of the gate.

Your last paragraph for a magic screen reader is that it simply doesn't exist, so in its absence you need to use a GUI FX that supports accessibility for it to be accessible. If all the screen reader can see is a rendered bitmap they're not going to be able to identify what's a UI control or how to interact with it, which group of pixels is decorative and which is functional or how it will be able to determine the difference between a real App and a screenshot of it? With the recent real-time AI powering "copy text from image" maybe a generic reader is going to be more advanced than what's historically been possible.

But I don't really know how screen readers work, I'm assuming they need to work in tandem with GUI FX's which is able to describe the purpose and roles of its different UI elements, so if you use native OS controls it's going to be able to know how to inspect different controls of running Apps and how to send events to them.

> If all the screen reader can see is a rendered bitmap they're not going to be able to identify what's a UI control or how to interact with it

This has been the status quo for decades of computing, but it seems quaint in this age of instant ML-driven image recognition. Anybody doing anything in the accessibility space with ML?

Apple: https://machinelearning.apple.com/research/creating-accessib...

I read your comment and agree. Without thinking it through, it seems that if every program provided a full CLI and a sitemap like file, then a screenreader should be able to integrate with said program.

I don't know much about the accessibility business, but my impression is that the screenreaders are all very expensive. Maybe providing a universal interface as described would level the playing field, maybe that's not desired (by the big brands).

> I don't know much about the accessibility business, but my impression is that the screenreaders are all very expensive. Maybe providing a universal interface as described would level the playing field, maybe that's not desired (by the big brands).

False. I have a print-related disability and I rely on screenreaders.

I use Orca on Ubuntu. However, the default voices for Orca are deplorable (by default eSpeak text to speech engine).

Most people do not know this, but https://oralux.org sells machine learning high quality voices (NeoSpeech engine) for $35 per voice at most which work directly with Orca. Here are the English text to speech voices: https://www.oralux.org/voice.php#english_american_english

This is an interesting idea for app-interoperability in general. It brings to mind Apple Automator, which I have no idea how works, but can sometimes be used to make Apps interoperate. It'd be pretty cool if every app did have a sitemap-like API spec that that mapped out the core of the app, and the UI sat on top of that.

A lot of the UI scripting on macOS is accomplished via accessibility APIs. If you’re interested, check out Accessibility Inspector[0], which ships with Xcode. You can inspect macOS applications in a similar, albeit much more limited way to a web browser’s interactive inspect element function.

You can also accomplish UI scripting via C/ObjC/Swift using the Application Services framework [1].

[0] https://developer.apple.com/library/archive/documentation/Ac... [1] https://developer.apple.com/documentation/applicationservice...

Exactly.

We should decouple the core of the application from the UI. Have a standardized interface which exposes all available functionality to the screen reader and allows it to talk to the core directly, bypassing the UI.

Stop doing what everybody does today: stop having screen reader depend on the UI (ie. aria-labels, etc). This problem can't be solved by adding "decorations" to existing UI components. It's best solved by exposing an entirely separate interface which lets screen readers (and other hardware) interact with the core of the application directly.

This sounds like a spec, a set of language frameworks and a design pattern waiting to be written.

If it's done right and standardized it'd be worth it for everybody since even people who don't need accessibility features could then create their own interfaces to the programs they use customized for their preferences and workflow

These APIs have existed for like three or more decades for desktop apps. That's partly why most desktop UI toolkits are based on tree of widgets: Mac and Windows's native accessibility APIs expect a tree

FWIW, the oldest accessibility API was Microsoft Active Accessibility (MSAA), which was added to Windows 95 in a service pack in 1997. Classic MacOS never had an accessibility API; the Outspoken screen reader for that OS had to rely exclusively on ugly hacks (e.g. patching Toolbox functions). OS X introduced accessibility APIs for Carbon and Cocoa in an early version; not sure which one. Unix-based desktop environments got an accessibility API, AT-SPI (thank you Sun), in the early 2000s.

well TIL, I believed accessibility to already be there by the late 80s, in NeXT workstations and the first windows versions but it looks like that was not the case.

Hi, the author here. Psst is definitely in alpha, and the CLI is indeed just an example (it's what I was using to test the core mechanisms). I'm sorry the accessibility is so bad now, but Druid, the GUI library, takes it very seriously -- which is not very common in custom GUI frameworks. So, fingers crossed, it should get better soon.

Re. command-line Spotify clients, there is ncspot[1] and spotify-tui[2], and if you use Linux, Spot[3] looks like a nice GTK experience.

[1] https://github.com/hrkfdn/ncspot

[2] https://github.com/Rigellute/spotify-tui

[3] https://github.com/xou816/spot

This is, unfortunately, a very common occurrence when GUI frameworks are concerned. For some reason, framework authors like to mislead and say that their framework is native, when it looks native but actually isn't. WX and SWT are the two notable exceptions here, they do indeed use native OS controls.

If you want to use a native GUI framework because of accessibility, check if it's as native as it claims.

You're comparing a person's side project to years of work on this in Chromium.

Seems rather like comparing someone's homemade bottle rocket to a Saturn V...

Your analogy doesn't make sense since I'm paying the same amount either way - so of course I'll choose the Saturn V.

Perhaps a better analogy would be whether you want a fast but fragile kit car missing some features, or a well-built but slower caravan.

To be fair, accessibility is not listed on psst's roadmap at all yet.

I'm intrigued by this client, but I'll wait for a beta before trying it out.

Fun fact about Spotify web's accessibility: Alt+left/right are used to both navigate pages AND go back/skip a song. It is truly an abomination.

I honestly think we need to rethink accessibility from the ground up. modern advances in OCR and machine learning should allow us to do accessibility entirely from the GPU output.

it'd take awhile to perfect but i think it'd ease the burden tremendously for software developers and those who need accessibility.

You're onto something here. As long as accessibility is the job of the developer, software accessibility will reamin "on the roadmap" for many projects. There's simply too much to learn, too many battles to fight, and accessibility tends to offer to little obvious and immediate value compared to other priorities.

Legislation is not really the answer, or we'd already have great accessibility.

In other areas, we've automated and outsourced it - CI/CD, security, infrastructure. Think how hard it was to manage an RDBMS cluster or version control repository 20 years ago, and now even a junior developer is able to have both within minutes. We have frameworks like React.

In that time, accessibility has actually got harder, because of more devices, technologies, network connections, and more people with a wider range of accessibility needs.

I'm not sure what it would look like. Maybe a new way of rendering content, maybe a new framework that puts accessibility as a first-class concept.

> it'd take awhile to perfect

Some of us can't wait. That's why I, for one, continue to advocate for developers to make their applications accessible with the currently available tools. It's also why I'm trying, with my AccessKit [1] project (which admittedly is taking time to get off the ground), to make it easier for GUI toolkits to implement the current baroque platform accessibility APIs.

I'm also reluctant to concede that we're doomed to reconstruct UI content and semantics probabilistically from pixels, when that information is already there somewhere in the app. But it may be the best long-term solution to the social problem of trying to get everyone to implement accessibility.

[1]: https://github.com/AccessKit/accesskit

Reply on news.ycombinator.com