Quote from Alan Kay about Knuth:
When I was at Stanford with the AI project [in the late 1960s] one of the things we used to do every Thanksgiving is have a computer programming contest with people on research projects in the Bay area. The prize I think was a turkey.
[John] McCarthy used to make up the problems. The one year that Knuth entered this, he won both the fastest time getting the program running and he also won the fastest execution of the algorithm. He did it on the worst system with remote batch called the Wilbur system. And he basically beat the shit out of everyone.
And they asked him, "How could you possibly do this?" And he answered, "When I learned to program, you were lucky if you got five minutes with the machine a day. If you wanted to get the program going, it just had to be written right. So people just learned to program like it was carving stone. You sort of have to sidle up to it. That's how I learned to program." - [1]
[1] - http://www.quora.com/How-would-Donald-Knuth-fare-as-a-compet...
I'm unfortunately cursed that I have a lot of trouble starting a problem till I really really understand all it's details and the details of my solution. I basically write down exactly what I want and am going to do on paper
I find that my coworkers that start with a vague idea of what they want (without a complete understanding of the system) but work in quick hack->fix iterations produce results significantly faster.
I don't have a huge sample size, but that's simply what I've observed. The person that finishes first is the person that starts typing first. The one that mulls over everything in their head might have a more elegant solution and a clearer git repo.. but they always finish last
In reality, writing a program in longhand is something everybody could do -- but because nowadays programming is mostly plumbing, you have to empirically test everything.
Anyway, the custom OS ("Waits") made good use of the Data Disc graphics system: it had a built-in interactive line editor, so that when you were in the shell, you could edit your command line (control-d deletes a character, etc., etc.) and see the result in realtime. (This was years and years before Unix got similar features in tcsh and bash and the readline library). All programs inherited this functionality automatically, so Wait's full-screen editor ("E") was simply built on top of it. (Again, years and years before emacs and vi, and all on a system with a per-process address space of only 256K words (about 1Mb), split evenly between data and code.)
So, to finally answer your question: while Knuth did spend his early years programming on punched-card batch systems (where you pretty much had to write out code long-hand before keypunching it), by the time he had started working on TeX he had been exclusively using a full-screen editor on a graphical display for many years.
Doing them on paper, he'd get to a point where he had a good understanding of his solution and was extremely confident that they'd work on the day!
I've never programmed on paper to any real extent but doing crosswords in pen certainly made me better at crosswords.