I think this ties in well with Erik Meijer's essay from a few days ago[1]. If you are trying to produce a framework for mathematical reasoning, it does not make sense to allow arbitrary exceptions. As a species, humans are pretty new to this whole computing thing, so I would agree we don't have it figured out yet. However, over the last 5000 years or so we have developed a shockingly effective toolkit for reasoning about arbitrarily-defined abstract objects and I don't think it is unreasonable to try to bring those tools to bear on a newer class of abstract object, that is to say programs. The cost of this is that we have to figure out how to formalize our ad-hoc methods. This will be painful, but every engineering discipline does it. In essence, formalized design methods applied to practical problems are the core of engineering practice in general. For example, I think software engineering right now is in the same place as electronic filter design was in the 1920's. We have methods that work, given enough design effort, but our results are brittle and it is very difficult to make changes to working systems without introducing problems.
[1] https://news.ycombinator.com/item?id=7654601 [2] http://en.wikipedia.org/wiki/Butterworth_filter#Original_pap...
Note: I am blown away by Sussman's talk every time I watch it, but I think throwing out math as a tool for understanding computing is very premature given its success as a tool for building and understanding abstraction, especially given that computing as a field started as an attempt to understand some deep questions about the foundations of mathematics.