Use Spreadsheets Everywhere

For example, Excel would be a lot more usable and maintainable for me if there was a way to make a special "data sheet" in which data types are forced to be consistent within columns and there was a concept of column names. Still GUI-based and user-friendly. That would encourage a logical seperation between data entry, data output, and computations. In my experience, the main challenge of helping users with spreadsheets is when they create spaghetti code that mixes data and computation together.

Excel thrived in banks, not because traders used it, but because IT/Quant people used it. It's so much more straightforward to build a UI in Excel than with React. Just type things in cells, then wire them with simple VBA buttons. Of course there are things you can't do, but for the purposes of a bank, it's very rare that you find one.

So if I am a quant, I don't need to go to the frontend guy, trying to explain to him what a "vega" is and why I would want to multiply by notional/vol to change the units. This is a huuuuge time saver. And you can change your UI anytime. Just open the file, add a column, click save and it's done.

At some point, banks knew that fat-finger-mistakes could cost them fortunes, and the lack of auditing was terrible. But they had to force traders to switch off their sheets by threatening them with internal fines (we'll charge you $2m for running things with Excel). So traders complied, but then they got into years-long projects, to create shitty shitty web apps, that couldn't do half of what Excel had built-in. Every change would need to go through an approval process, and it's unclear whether these systems had less bugs than spreadsheet did.

The first thing they wanted when the project was done? "Give me a button to export to Excel".

For the shameless plug part, I am building a tool to try and bridge code and spreadsheet (https://www.jigdev.com). Lmk if you have comments/suggestions.

Everyone uses spreadsheets, including developers. I use them. I'm not going to spend 50 hours coding when I can create a spreadsheet in 5 minutes.

Of course, the issue is when things get complicated, as the author and others here have noted.

Creating a CRM from spreadsheets is pure madness, for example, when there are so many other options that exist, without having to custom program.

For me, whenever you have a one-to-many or many-to-many situation, that's where spreadsheets, to me, fall apart.

I personally have used spreadsheets to do my finances, but only because I was too lazy to scope out different bookkeeping systems. I'm fairly expert in accounting, too. I've used a LOT of different accounting systems, and installed and trained people on them. But there's no way that one can get up and running as fast as one can by using Quickbooks or other accounting systems, in terms of all the report features, etc.

Pre-defined apps are fairly unchangeable, but unless one has critical information that depends on a custom solution, it's better to shoehorn your business into a pre-existing app. I do admit that. Very few small and medium sized businesses require that, though, I've never seen one yet that can't use a prior existing solution. Though there might be, but only very, very small percentage, like, less than 1%. However, a large Fortune 1000 enterprise could have stuff they need custom programming, because of the scale. But I don't have any experience with enterprise organizations so I can't comment on that. Maybe SAP or whatever is good for them, I don't know.

The only situation I can think of where you should discourage their use is in those situations where you know for sure in advance that the application is going to outgrow the spreadsheet very soon and/or immediately. If you are building a new microservice that will serve a million customers per day, don't make the mockup in a spreadsheet.

Consider the output of your average audit. You'll have tables of findings, each of which needs a due date, a risk rating, a description of the problem, a description of the solution, auditor notes, customer comments, responsible party assignments, and so forth. (Yes, those would eventually go well in a tool like Jira, but that's for later--this is coming out of an audit visit.)

From a data standpoint, putting those in a spreadsheet makes sense: you can now order the findings by date or risk rating, hide ones that don't apply, cross-reference findings between visits, and so forth. However, from a text perspective, it's awful: the descriptions might run to multiple paragraphs, comments and instructions need more complicated formatting than just "bold or italic", some fields should be constrained on content while others need to be free-form, and so forth. All of those things work much better in a Word table than in Excel cells, but putting the content in Word utterly removes the ability to data-manipulate. So you wind up either creating some Frankenstein hybrid solution or with crushing one perspective to satisfy the other.

If Microsoft wants a win for Excel, making it an order of magnitude easier to deal with free-form text in cells would be an enormous step forward.

Parsing them is actually very complex. You just have to hope your library can handle everything in excel. If you're a cli native I'm not really sure what you do.

Spreadsheets don't play very will with source control. If the underlying format was text and every cell and formula was on a new line it would work out ok. As it is now, merging them is very cumbersome....impossible for the laymen that might be the main user of the sheet.

By going with a speadsheet and not something like SQL, you lose a lot of rigor.

There's probably a few more needs but these are top of mind for me.

All that said, I can see issues with errors that are hidden by spreadsheets but those can be handled with good spreadsheet design (i.e. NEVER EVER hard code anything into a cell) and they simply just need a way to do GIT type version control natively. We recently got the ability to have multiple people work in the same spreadsheet at a time if on MS Teams, but there's more work to do there.

Minor quibble: it's an Apple III in the ad, not an Apple ][ as the post says. I remember running Visicalc on my Apple ][ but it wasn't until Excel appeared that I realized how seductive spreadsheets can be.

When we had no front-end devs available - we used Googles sheets API, connected our spreadsheets to our production DB, and did all the input of raw data, and ingestion of outputs automatically. User interaction happened within the spreadsheets.

When FE devs opened up, we eventually built the UI to replace the spreadsheet (Which is a very hard task, spreadsheets are good at what they do). But by this point, we knew exactly what we needed in the ux and had iterated on the ux multiple times.

Framework was pretty much Emacs for the office, and such was its power that unlike contemporary spreadsheets and word processors, which were usually positioned as productivity tools for generating business documents and reports, Framework was explicitly marketed as a decision making tool for executives. It did well in markets such as Europe where Lotus hadn't taken over, but it just about died when the Windows era started.

[1] https://docs.microsoft.com/en-us/office/dev/scripts/develop/...

Once you are able to read & write excel documents with a piece of software, you can do some pretty fucking incredible things.

Imagine being able to click a single button and download a total configuration output for a customer's environment (maybe 20-30 worksheets auto-generated in seconds). You can then email this human readable document to the customer for modifications. You then feed this back into the system to load their adjusted values (there is a diff/check-in report to confirm first).

The reason we and our customers like this approach is because there is a lot of configuration where we need to compare lists of things and slice parts of one thing into another. It makes replicating success absolutely trivial. Being able to style the document is a much bigger benefit than you would probably think at first. CSV is trivial to employ, but it is very constrained on this front. For a developer who is familiar, color & layout doesn't move the needle much. For a customer who has no clue how the back-end works, these things make all the difference in the universe. Things you can't edit are grey background, things you can are green, etc.

It's the user's decision that's usually wrong. Not the framework in which the solution is built.

For example, at Budibase [1], we've found a high percentage of our use cases are companies upgrading their in-house spreadsheets to applications due to several reasons: Volume of data Lack of auditing Lack of control Accessibility

[1] https://github.com/Budibase/budibase

Recently, we've started to see a major shift away from spreadsheet models to spreadsheets being used as data dumps and then being uploaded into third party software. Usually this "software" tends to be nothing more than some SQL and python scripts behind a fancy UI. I would personally prefer the banks to continue using Excel sheets and then pay for their employees to learn basic python and SQL but that's too simple and the SaaS sales guys have their claws firmly embedded in the upper levels of most orgs.

At https://www.dronahq.com we support 1. users to connect to their spreadsheets and build applications on top. or pull data from multiple datasources e.g. spreadsheets and support ticket system and build custom tool.

2. We understand people love columnar database like spreadsheet or airtable.com, hence we offer an built-in database called 'sheets'

3. We support most excel formula and functions for binding data or writing complex business logic i.e Logical Functions, Date and Time Functions, Text Functions, Math Functions, custom functions

However, I think spreadsheet programs are still way to complex for the vast majority of people. So many people are so math phobic that they won't even think of opening a spreadsheet.

One thing I've done with a spreadsheet a few dozen times is take the outputted CVS file my learning management system gives me in the form of last name, first name and convert it to first last in one cell. I still have to find the bookmarked answer I found years ago and cut and paste it in and I don't really understand how it works.

Spreadsheets are broken. It's easy to be at error due to calculation issues, which you'll only find out when manually going over it. It's a calculator that can't properly calculate. Many business have made huge mistakes due to it.

Never ever use it for financial calculations.

It should be interesting for anyone who's done a lot of thinking* about the relationship between spreadsheets, no code, and custom software. Check us out: https://stackerhq.com

* if this is you, and you're really interested, we're hiring. Email in my profile :-)

However, if your use case is narrower (e.g. only use CSVs), there are more suitable tools.

As a person who knows how to code, using a real programming language or SQL is way way better.

To that end, I've built a Desktop app that enables you to work with CSV using SQL here: https://superintendent.app

The main problem is when you embed a table in a sheet and use it as you would in a database. It is far too easy to overflow the maximum spreadsheet size, or overwrite data... or worst of all sort some but not all of the columns.

I want to like LibreOffice Calc but it's just too slow.

It's interesting to see non-excel/google sheets products use the same interface to good effect. Examples don't come to mind but I'm sure there are some.

I am a CFO that has used spreadsheets my entire career (30+years). They can definitely have a vital role in any company and can also cause incredible problems.

A few thoughts that were not mentioned in the article:

   1) in many companies the users of spreadsheets are not allowed to use anything more powerful than spreadsheets (or maybe MS Access which causes more problems).  To get a SQL/Python (or other programming language) solution requires submitting a request to IT and then waiting two years to work up the priority queue.  This is not a complaint against IT and the priority process.  The reality is most spreadsheets do not start out as important enough to warrant high priority.  
   2) Spreadsheets are great containers.  Not only can they hold a model, but also the raw data, and even emails regarding the model/inputs/etc.  This makes them very handy for keeping everything in one file regarding an analysis.
   3) IME, most significant spreadsheets get rewritten as they evolve, especially when they change hands.  This is both good and bad.  While the rewrite allows correction of built up badness, it is rarely done with the discipline one sees with refactoring in more traditional code.  Moreover, when a hand off happens, the rewrite is generally done because the recipient does not fully understand how the existing model works.
   4) Large spreadsheets cannot be effectively audited outside of locking/password protecting non-input cells. Very rarely is this actually done.  As a result, you get the scenario the author describes of hard-coded values in cells that once contained formulas.  These are almost impossible to find in worksheets with large numbers of tabs and thousands of formulas.  
  5) Spreadsheets generally do not use true version control.  For most of my career, one simply used "File Save As" to create a new version.  In the early days it was due to a fear Windows would crash and you would lose your work.  Later, it became a way to step through changes and determine the impact of the change.  While this method provides some ability to go back to a prior state, people really do not track properly what each version contains and what are the true differences (would love to find a way to use Git effectively on spreadsheets).
   6) Most spreadsheets do not have test cases.  They certainly do not increase the test case universe as modifications are made.  As a result, cannot tell if things break when changes are made.