Hacker News Reader

Comment by mtrovo | Hacker News Reader

> I've mentioned that programmers are far too dismissive of MS Excel. You can achieve a awful lot with Excel: more, even, than some programmers can achieve without it

This is one of the most underrated topics in tech imho. Spreadsheet is probably the pinnacle of how tech could be easily approachable by non tech people, in the "bike for the mind" sense. We came a long way down hill from there when you need an specialist even to come up with a no-code solution to mundane problems.

Sure the tech ecosystem evolved and became a lot more complex from there but I'm afraid the concept of a non-tech person opening a blank file and creating something useful from scratch has been lost along the way.

View on HN

Reminds me of the investment banking dev cycle in the 2000s:

* trader writes a pricing "app" in Excel

* trader discovers MS Access db

* traders (plural) start copying around Access db files

* problems

* "IT" gets involved

* convert MS Access to oracle or sybase

* write some server process(es) in C++

* write some replacement front end (spend months arguing over best grid component to replace excel) in C++/MFC

* trading system emerges...

* rewrite in C#, Java

* etc.

problem as described to me is that excel starts being used for regulated processes and it's not well auditable, access controlled, changed controlled, tracked, etc etc. Then people need to implement the exact same process across departments and they're all using a separate excel sheet and they all submit different numbers. becomes a huge mess and so much more complicated and expensive systems become commissioned.

This describes what I've seen happen with Excel over and over again. I'm curious if the use of collaborative Google sheets could be a fix for this? Something where a portion of the sheet could be shared globally, but the rest of the document would be local to the instance working on it.

Fun story: I was at a bank that used Excel for everything. As you say, there came a complaint from the auditors that it's not well auditable, and there needed to be "a system".

Solution: the bank put together a system that constructs (from Excel templates and the bank trading data and market data) Excel spreadsheets from scratch every day, then used those for the calculations, and stored them. But now it was "a system", so all good.

Well you can audit the code that generates spreadsheets, which seems to solve the audit problem. Kind of like I prefer reading a Dockerfile that builds a program from the GitHub repo, rather than downloading a pre-compiled package I can't trust.

sounds like a great system. we have something similar where we put excel in and out but doesn't sound as slick as that. on top of the system there is access control, versioning and such. the data gets approved and then stored in the backend to feed the regulated process.

There's an excellent example of this phenomenon in the JPM "London Whale" report where -- at various points -- poorly maintained and validated spreadsheets appear as minor villains in a $6.2bn loss.

The jargon for this is "user-developed application," and auditors do keep an eye out for these. Banks, from what I've seen at least, typically have some process to document these as they come up, replace them with supported solutions, and retire them. At least, that's the "happy path," where people are willing and able to get all that done before a big-three auditor comes in and tears you a new one.

Plus, a spreadsheet is basically purely functional (unless there's mucking around in VisualBasic), and has a beautiful dependency graph and calculation engine! (And that is a big part of what SecDB/Slang/Bank Python brought to the table.)

I think part of the problem with Excel (or clones) is that you can do so much haha. Its such a powerful tool, that you end up doing things in it that it really wasn't optimized or designed for and managing the change history in excel is pretty tough.

But for 95%+ of analysis you really cant beat it.

Very true. I often prototype algorithms and things in google sheets. One time I had backpropagation working in there, with a little button to process the next "row" of training samples.

I used to work for a very successful company that produced mobile games. The entire logic of the game, the rules, etc. was all in Excel

So Excel spreadsheets were deployed on mobile devices? What about the runtime?

The problem is sometimes analysts turn into shadow BI or even DE who only know Excel. They know Excel so well that they create a whole monstrosity in Excel. MSFT has been sort of encouraging that too by introducing some Power BI feature and now Javascript into Excel.

Reply on news.ycombinator.com