Hacker News Reader

Comment by londons_explore | Hacker News Reader

It's time screen reader tech moved towards automatic OCR.

It's a losing battle trying to get every technology and web service to put in all the right aria tags - imagine imgur for example - there is no way you could make those all those animated gifs and mspaint memes properly accessible - instead, a neural network should look at the screen like a human does, and describe it.

The screen reader should say "There is a gif of a squirrel falling off a shelf, and it has a share and an upload button"

View on HN

Facebook automatically creates alt tags for images and describes them with text such as "three people outdoors with a tree." As someone who's blind I found it interesting for five minutes and have not visited Facebook again. I have been totally blind since birth, and I don't think any amount of textual description will make me interested in scrolling through a photo stream. I could see this being different if you lost your sight later in life. OCR in it's current state is a bad idea as a primary way to get information when using a computer. My screen reader allows me to use OCR to find out what's on the screen. It works OK for specific situations such as an inaccessible installer with a couple controls. It falls down if I were to try and use OCR to read Java code.

I'm sympathetic to the GP's point. Every time I read about yet another custom GUI toolkit, I want to scream, because I know it might lead to more applications blind people can't use. If screen readers could really see the screen for themselves, instead of having to be spoon-fed accessibility information, then custom toolkits with no accessibility support wouldn't be an issue anymore.

But, as you said, OCR isn't perfect. I'm not surprised that it still has inaccuracies when recognizing code in particular. Also, computer vision takes a lot of computing power. I'm going to be sad if someday it ends up taking a powerful GPU, possibly running on a remote server, to enable a screen reader to do the things that a 90s computer could do without breaking a sweat via programmatic accessibility APIs.

Reply on news.ycombinator.com