I don't think that Excel was ever in need to be defended. It's one of the best analytic tools for tasks of proper scope. It was also the first tool that we had to learn and master in bioinformatics class before writing even a single line of code.
It's fine if you want to keep a small budget or a quick bit of work with numbers, but it's awful for dealing even modest amounts of data and here's why:
Nothing you do is captured and everything is destructive.
Any time someone edits their spreadsheet they are irreparably altering the data without real documentation. Excel encourages moving, shaping, dragging, and futzing that may be easier, but ultimately can cause major problems because there's no audit trail. Ever try and replicate something modestly sophisticated that was done in Excel? Good freaking luck.
There's nothing wrong with Excel for some basic graphs and calculations. The problem is when people start to use Excel as a data base or as a serious tool for analytics (like some of those business folks).
Excel is as ubiquitous as it is because the mental model of a spreadsheet is very simple and easy to wrap one's brain around. Here's a grid of cells, type stuff into them. Boom, you're an Excel user. (It can do a lot more than that, of course, but many many many people never realize that and still get lots of value out of it. Which is kind of remarkable, really.)
Proper databases are much more powerful, but none of them present the user with an interface whose basic metaphors are as immediately obvious as Excel's. You have to learn about tables, data types, relations, etc. to get anything done in them. Most of that stuff comes up with Excel too, but you don't need to know it to get started. The Excel user's first baby steps are much smaller than the database user's are.
Given a choice between a flawed tool that's immediately understandable and a perfect tool that requires learning, people will always choose the flawed-but-understandable tool. So they go with Excel.
I think much of the explosion in data science jobs these last few years isn't really about "big data", but rather the fact that your typical corporate analyst does not have the experience to tackle mid range data analysis jobs because the only tool they have under their belts is Excel. What really is needed is a larger number of people with experience in statistics, scientific computing (Python, R etc..), and database systems. The current tools we have are fine, they just need to be used properly (well, we should always strive to improve our tools, but you get the idea).
Do you think Excel with built in version control would help in solving the problem? I remember reading about some company that was trying to do that but no idea where they are now.
Every tool has a scope that it has to be used on. If you use Excel in situations where you need team collaborations, stability, or other really advanced stuff, you're doomed.
However, if you just need to check a quick hypothethis, or mock up a solution, it's really the best tool for the job.
I guess our fund isn't "hardcore":)
I think the finance industry runs on Excel.
I work in aeronautics and everyone who works on a computer maintains some Excel spreadsheets. It's everywhere but so few people know how to use formula, pivot tables, etc, that it has become a real competitive advantage to master them.
You did remind me of this amusing paper from a few years ago though: http://www.biomedcentral.com/1471-2105/5/80