How a Blind Developer uses Visual Studio [video] | Hacker News Reader

How a Blind Developer uses Visual Studio [video]

youtube.com

185 pointsnevatiaritika63 comments

I always wondered how someone could use text-to-speech effectively, especially with a programming language full of symbols with long names such as 'semi-colon', 'underscore', 'left parenthesis' etc. The answer it seems is to speed up the voice tenfold! Honestly incredible he's fine-tuned his other senses well enough to be able to code at a reasonable speed again.

I was at a web design conference once with a talk by a heavily vision-impaired guy. I was amazed at the speed when he went to his IDE to fix a bug. I couldn't make out a word it was saying and he didn't seem slower working with it than I would be.

I'm glad you mentioned that. After watching this it made me wonder if this is normal, or if this guy is some sort of auditory wizard. It seems like an awful lot of work to master, but maybe not if it's a skill you're using everyday.

> After watching this it made me wonder if this is normal, or if this guy is some sort of auditory wizard.

The "visual cortex" area is not deactivated but gets re-tasked with non-visual processing tasks, even in "late blind" people (people who lost their sight after age 12): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3667661/

Considering the amount of visual information the sighted process more or less effortlessly, that's a pretty significant amount of processing capabilities which can get retasked with e.g. auditory processing.

I wonder if any of it is also being directed at just being better at programming.

> The answer it seems is to speed up the voice tenfold!

That seems very common for systematic text-to-speech users, I saw a video a few months back of a lawyer (?) demonstrating ios's accessibility (rotor, voiceover, blind curtain, …) and the TTS speech was so fast it sounded like white noise to me.

That would be interesting to watch. Do you have a link for that?

I believe this article from David Pogue is the one:

https://finance.yahoo.com/news/david-pogue-on-iphone-voiceov...

That's exactly the one, thank you! The relevant bit is at 4:10

Sadly no, I tried looking for it but I haven't managed to dig it back up.

I watched the video twice, and I picked up a lot more the second time. At first, it seemed like he had the speech set to "incomprehensible chirping".

I'm still impressed at how he found the syntax error, though.

One of the company I worked with had to design an audio interface for blind people. I proposed something relying on several stereo located sounds of different pitches and intensity. I was told this would be too hard to use for a normal user. I wish I had such a video at the time to show the kind of bandwidth a blind person is able to get from audio alone.

It's not only sped up, it's using a lot of abbreviations.

Here is a link to the Stack Overflow answer mentioned in the intro portion of the video:

"How can you program if you're blind?" | http://stackoverflow.com/a/453758

I met two blind programmers. The process they used was exactly how it's described in this video.

What was odd was that:

1 - they were given the worst place in the room: no windows, literally facing a wall. Took some time to realize why they didn't complain.

2 - The computer had monitors. Then I learnt that Windows (don't know what version it was) wouldn't work it no monitor was attached.

As a blind programmer who uses a laptop I still got a second monitor that was not currently being used by anyone. My co-workers thank me when they don't have to hunch over to look at my laptop screen when going over code.

two thoughts:

1 - does sun light have no physiological/psychological value for blind people? the sun light might still be captured on the skin or even in the eyes and trigger relevant biological processes.

2 - having a monitor is also important for a blind person when working in a team.

I'm totally blind and have what I call no usable vision. I can see light if it's bright or the sun is shining. I can't tell anything about the environment from my vision though other then if there is bright light or not. Anecdotally from talking to other blind people the ability to see light helps set your body clock. Although I keep a fairly normally sleep schedule I know blind people who have no light perception and have a lot harder time keeping a normal schedule with out medicine.

The vast majority of visually impaired and legally blind have some degree of sight or at least light perception. The tiny percentage that does not still needs/like sunlight for vitamin D etc, but you don't get much sunlight form a window.

For those wondering: you can get lots of sunlight, but not any UVB (https://www.ncbi.nlm.nih.gov/pubmed/19614895: "All glasses totally blocked UVB radiation."), and that's what your body needs to make vitamin D (https://www.vitamindcouncil.org/about-vitamin-d/how-do-i-get...)

Quite a few legally blind people have some remaining vision that is both useless for them and extremely annoying, so they wear sunglasses, and prefer not looking towards light sources.

So, this may have been by choice.

The idea of someone using a computer with no monitor seems to bizarre.

I know blind people wouldn't need one, but somehow the lack of it brings up the wow factor.

I imagine many would benefit if we had a smart developer assistant where it could have a higher level understanding of programming.

"Jarvis import library foo" - could be said from anywhere in file

"Jarvis jump to file bar.c"

"Jarvis declare var current temperature equals zero" - var currentTemperature = 0

"Jarvis write for enumeration on var word over var name list"

"Jarvis build and run" or simply "Jarvis run"

By the way, I know many people don't like to talk to a computer, can't imagine how it could be faster than typing, etc. However, I believe that once an assistant is done right, more than a few people would be converts.

And then one day there will be a generation of programmers who have only known coding via a voice-driven developer assistant. Imagine trying to interview such a person.

"Please stop shouting at the whiteboard. It doesn't work that way. I need you to pick up one of the dry-erase markers and write your algorithm to invert a binary tree."

"What? Who codes with their hands? I need my assistant!"

Would that be a bad thing though ? Assuming assistants are ubiquitous. We don't use punch cards in interviews today. Even today whiteboard interviews are sometimes not the best method to fill certain roles.

"You have to use your hands? That's like a baby's toy!"

IMHO I don't think it would fly. The most difficult things of programming are too abstract for that.

You may be interested in the guy that did all his coding with voice for a while because he couldn't use his hands, and still uses it to this day about half of the time. https://www.youtube.com/watch?v=8SkdfdXWYaI

Yes, I already know many people don't think it will work. I have that url, along with a few others in my Programming by Voice notes:

https://github.com/melling/ErgonomicNotes/blob/master/progra...

No one said that you have to do everything. Start with what's possible then iterate.

If I'm reading or thinking and leaning back in my chair I often want to say or gesture a command to scroll up or down for example, it happens more often than not when reading long articles.

(btw, if anyone knows anything about software/hardware to do that? let me know)

It would good to have the options to voice or gesture command a desktop computer.

I used to do this with Dragon Dictate. It worked very well; I recommend it.

I created speech bindings for my most common navigation tasks:

- "Down" -> presses PgDn

- "Down a bit" -> presses "down" a few times

- "Switch" -> presses Cmd+Tab (next application)

- "Backtick" -> presses Cmd+` (next window)

- "Apple L" -> presses Cmd+L (focus address bar)

Latency was good enough. Bindings have a higher priority than dictation, so if it hears one of your custom bindings: it will immediately action it instead of waiting to hear a full sentence.

Mac version was mostly only reliable for triggering keyboard shortcuts. Dictation of sentences worked pretty well, but there were (many) significant bugs that meant it was not generally viable.

I eventually stopped because of one of the following (can't recall):

- OS updates broke it - lost the license in a computer switch - hands got better

For basic gestures, an apple magic mouse is pretty good (also works in windows).

Almost the identical use case - I sometimes sit back and use gestures to scroll, swipe browser back, swipe between desktops.

An external trackpad would also do the trick, but the mouse is ergonomically nice for when you're stretching or leaning back.

Voice input is an option when other types of input are not.

In the same vein, I am learning morse code. Any resources helping with this would be appreciated.

I would like to use morse code for silent, no-look I/O on my phone. I'm not sure vibrations can be quiet enough; might have to use something special-purpose plugged into the headphone jack.

You could design a keyboard for it on Android at least.

Make the whole keyboard area a blank "tap zone" where you can just use any digit to tap text into an input field.

I looked and this has already been done a few times!

https://github.com/gitonwithit/MorseKeyboard

https://github.com/whymarrh/morse-code-keyboard

https://play.google.com/store/apps/details?id=com.Input.Mors...

For learning: Morse-it app. Not sure if it's on android, but it's a great learning tool with lots of learning methods.

This guy took the first step. Its a great watch even though its kind of long

https://www.youtube.com/watch?v=8SkdfdXWYaI

I'm sure someone on HN is seeing this right now and has already started working on it :). Especially with the advent of Google Assistant SDK and the likes, it's clearly very much possible now.

The crazy part to me is working in an office where 50 people are simultaneously saying code out loud.

Or my wife ready to kill me as I'm saying code while she's watching tv. :)

Rather than my usual tirade against open offices, I would point out that options for subvocal voice pickups exist like throat mikes. You could speak softly enough your neighbor couldn't hear you, and they also attenuate external sounds quite well.

So basically you'll be trying to think above a constant, low-level din of unintelligible murmuring. Not unlike having a brown noise generator in your office. That would drive me insane.

Well, when I say "your neighbor can not hear you", I mean that literally. Not mumbling, just talking at a sub-audible volume.

And, IMO, even mumbling would be better than the current open office constant flow of loud conversations (one of which I'm trying to drown out with headphones this very moment).

Drop the prefix keyword and then we might start talking about "smart".

Oh, and it must work locally instead of sending everything to the cloud.

Yep, there are demands you can certainly make after the problem is solved.

For now, it would be helpful to many if there were no such limitations. In other words, comments like "Make it even smarter and it can't go to the cloud then I might use it" can safely be ignored for now. There's a market for an MVP that does a lot less.

This is great. I always wondered how blind developers managed to navigate IDEs at a decent pace.

I can't help but wonder how Saqib Shaikh navigates through log files for large applications though. Log files usually generate a large amount of noise in the form of time stamps, thread names, log levels, stack traces, etc. that overtime I've learned to "scan" instead of read to find what I am looking for. I would love to see a video of him navigating through a log file to see how he does it. Maybe just lots of string searches?

To be fair, sighted developers have to filter all that noise out too when we slog through logs visually.

Damn, he works faster than me... the only thing I could have done faster was removing the "

Edit: wrote semi colon instead of " double quote

I know it's no where near the same thing. But this reminds me of when I broke my arm very bad as a kid, and was in a cast without the use of my fingers. I learned how to play Mario Brothers and Zelda with just my left hand in a few weeks. Besides a few key combinations, like working up speed in Mario, I ended up being better with one hand than two.

I imagine even that fix would have been quicker were it not for on-stage nerves and the crowd noise distracting from the TTS

And many modern languages don't have semi-colons. Wonder if they can be removed from C#?

I think he means the extra quotes that was causing syntax error.

Oh, that one hanging out at the end of the line? Hopefully, a better assistant could suggest that as a fix then you'd simply have to say "fix line 11", for example. It has got to get increasingly difficult to find even simple syntax errors in a lot of code, if you are visually impaired.

In my first developer job, I worked with a blind developer. Back then it was green screens, ksh, vi, and a screen reader. That was over twenty years ago.

I'm surprised the process is very much the same today as it was then.

What screenreader is he using?

Sounded and behaved like the default narrator that comes with Windows. They all tend to rely on the same APIs, though, so what they “see” of the screen or the focused control is very similar.

Listening to the beginning of that video as a blind programmer I'm 99% sure he's using Jaws for Windows. http://www.freedomscientific.com/Products/Blindness/JAWS I program in Java and Eclipse for my day job but have found Visual Studio to be really nice to use and accessible with Jaws.

why can't we code like this "remove that comma on line 3" or "move line 4 inside the braces above"? surely it would benefit everyone?

you could even ask "which lines have if statements?"

for something like this wouldn't you be talking to an AST rather than a text editor?

How is Xcode for this?

In my experience it's pretty much impossible to get any kind of UI done in Xcode, at least as of Xcode 7. I'm a totally blind programmer who has a Macbook Air that I run windows on most of the time. I'm not sure if my issues with Xcode were do to a lack of experience with Voiceover on the Mac, lack of knowledge about iOS programming, or lack of accessibility in Xcode. It would be nice if Apple could come out with a basic guide targeted at Voiceover users to create a simple app that allowed you to enter text into a text box, click a couple of buttons and have something happen, etc. I've had a really ahrd time following any guides online to get started with iOS programming. Not sure what other blind developers experiences are though.

Visual Studio seems like an odd choice...

He works at Microsoft.

But seeing "Visual Studio" and "blind" together did make me look twice.

Wouldn't the IDE with the largest financial backing seem like the best choice? Making complex things accessible isn't cheap or easy.

And why is that? o-O You think you can rename some class method Add from some project to Enqueue faster in vim?

Reply on news.ycombinator.com