True, he doesn't seem afraid to take big leaps. Brilliant, visionary essay. But I don't think it's unrealistic. In my day to day (game dev) I spent very little time optimizing (and very rarely need to write any sort of bit shifting), but instead a huge chunk of development is spent slingshotting variables through loops into abstracted rendering directives. The hardest part in my job is by far understanding and visualizing program flows in my head. From the essay:
Wait. Wait a minute. Were you trying to answer those questions by doing arithmetic in your head? The computer somehow drew that picture, so the computer must have calculated all those scaleFactors itself. Are you seriously recalculating them in your head?
Sadly, yes, very often I am. Depending on the mood I also use a convoluted combination of notebook + pencil, breakpoints, printf statements, and/or isolating the problem by working on a separate program with the problem simplified. And having aids - like some of the great solutions suggested like the timeline and other state/flow displays - would give me tremendous gain today, if they could be integrated in my toolset. I'll give you that I imagine it being quite a bit more challenging technically to make it work with my C/C++/Objective-C environment, but with other more dynamic languages - like Javascript or as suggested: Clojure - I imagine it to be much easier.
This article is by the way, most likely the best argument I've heard so far for me to move to a higher level language. I like how Bret Victor talks about language (and API - see the autocomplete argument) choices as enablers to better environments; I think we instead usually think of language choice as something that mitigates how hard this work is to begin with with the tools we have (see the time and energy we spend in in this community on language choice). If I had access to tools offering this level of abstraction but had to adopt a language that I didn't particularly like, I think the latter would then become be the least of my concerns.
Another curious example is Erlang, which lets you update the code responsible for some behavior (I mean of an object) live, without any hassle at all. Then there's ClojureScript, which lets you do the same in browser (I think, don't know it very well).
Foundations of what Bret says are not new, and they are not purely theoretic either. The programmer of 2012 could have been using them for years if he wanted to.
I think Bret is right when he writes that technical possibilities are not a problem - programmers mentality is. Both Lisp and Smalltalk programmers would welcome - I imagine - Bret ideas without a shred of hesitation, because they are working in a similar way since times immemorial. It's just that there are so few of them and Bret wants to influence programming at large - that's why it seems to be difficult or novel.
The geometry example is chosen because it's easy to make a mapping between the space of function inputs and visual outputs. And each parameter is independent of each other.
Khan Academy has already implemented some of this for JavaScript, running right in the browser.
http://www.khanacademy.org/cs/drawing-bonus-rotation/9064481...
Try clicking on the coloration functions to see previews, or sweeping with the mouse to change the values.
As for the more advanced features, many languages exist today which make this quite possible, at least for teaching tools. Even well-commented Java has the kind of typing and documentation culture that would allow you to implement a lot of this today.
In Bret's video, where hovering over a parameter would show what exactly would change – that requires either machine-readable metadata (i.e. x position of top-left corner of the shape) or additional programming to make available. Javadoc as it exists is just a semi-structured and very thin wrapper around HTML. Not really much a computer could do anything with.
The general problem I see with Bret's approach, while it works very well for restricted programming environments intended for beginners and learners, it falls short for more complex things. But then again, those of us who know half the language framework by heart anyway won't probably need as much guidance or fiddling around. Still, it requires augmenting each and every function with a piece of visualisation code or enough metadata so a development environment can apply the visualisation itself.
I'm absolutely so surprised by the lack of imagination here at HN. This is startup ideas gold right here, and none of you seem to have the mental capabilities to dig it out and do something with it. You all want to find something wrong with it.
Ever since I've been tracking Bret Victor, my head has been spinning with tons of practical ideas we can implement today to make us more productive at programming. Light Table is a fantastic start. What's the problem here, guys! You guys just don't want to admit the way you've been coding thus far will soon be obsolete and you've been wasting your time. Yes, your skills will eventually be worthless.