Really what Victor is complaining about is the web. He doesn't like the fact that we are hand-coding HTML and CSS in vim instead of directly manipulating spatial objects. (Although HTML is certainly declarative. Browsers actually do separate intent from device-specific details. We are not writing Win32 API calls to draw stuff, though he didn't acknowledge that.)
It has been impressed on me a lot lately how much the web is simply a distributed Unix. It's built on a file-system-like addressing scheme. Everything is a stream of bytes (with some additional HTTP header metadata). There are bunch of orthogonal domain-specific languages (HTML/CSS/etc vs troff/sed/etc). They both have a certain messiness, but that's necessary and not accidental.
This design is not accidental. It was taken from Unix and renamed "REST". The Unix/Plan 9/REST principle is essentially the same as the Alan Perlis quote: "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures." [1] The single data structure is the stream of bytes, or the file / file descriptor.
For the source code example, how would you write a language-independent grep if every language had its own representation? How about diff? hg or git? merge tools? A tool to jump to source location from compiler output? It takes multiple languages to solve any non-trivial problem, so you will end up with an M x N combinatorial explosion (N tools for each of M languages), whereas you want M + N (M languages + N tools that operate on ALL languages).
Most good programming languages have the same flavor -- they are built around a single data structure. In C, this is the pointer + offset (structs, arrays). In Python/Lua it's the dictionary. In R it's the data frame; in Matlab it's the matrix. In Lisp/Scheme it's the list.
Java and C++ tend to have exploding codebase size because of the proliferation of types, which cause the M * N explosion. Rich Hickey has some good things to say about this.
I would posit that Windows and certain other software ecosystems have reached a fundamental scaling limit because of the O(M*N) explosion. Even if you have $100 billion, you can't write enough code to cover this space.
Another part of this is the dichotomy between visually-oriented people and language-oriented people. A great read on this schism is: http://www.cryptonomicon.com/beginning.html . IMO language-oriented tools compose better and abstract better than visual tools. In this thread, there is a great point that code is not 2D or 3D; it has richer structure than can really be represented that way.
I really like Bret Victor's talks and ideas. His other talks are actually proposing solutions, and they are astounding. But this one comes off more as complaining, without any real solutions.
He completely misunderstands the reason for the current state of affairs. It's NOT because we are ignorant of history. It's because language-oriented abstractions scale better and let programmers get things done more quickly.
That's not to say this won't change, so I'm glad he's working on it.