Hacker News Reader

Comment by corry | Hacker News Reader

Always overlooked point in these pro/anti-spreadsheet discussions:

A spreadsheet gives you a DB, a quickly and easily customized UI, and iterative / easy-to-debug data processing all in a package that everyone in the working world already understands. AND with a freedom that allows the creator to do it however they want. AND it's fairly portable.

You can build incredible things in spreadsheets. I remain convinced that it's the most creative and powerful piece of software we have available, especially so for people who can't code.

With that power and freedom comes downsides, sure; and we can debate the merits of it being online, or whether this or that vendor is preferable; but my deep appreciation for spreadsheets remains undiminished by these mere trifles.

It's the best authoring tool we've ever devised.

EDIT TO ADD: the only other thing that seems to 'rhyme' with spreadsheets in the same way is: HyperCard. Flexible workbench that let you stitch together applications, data, UX, etc. RIP HyperCard, may you be never forgotten.

View on HN

Right? Spreadsheets have such a low barrier of entry. I use Google Sheets in a middle of a farm field to enter plant growing records on my cell phone. I don't even have good reception at times so the sheets are in offline mode; they'll synch when I get back to the house. Crappy, crummy records are oftentimes better than no records. Farming is data-heavy and data-starved at the same. A learning cycle is a year, and one year is not like the other one.

> especially so for people who can't code.

And for those who can, Appscript gives your spreadsheet super powers.

For those who don't know, you are not stuck with writing JS in the Appscript integrated web IDE that comes with Google sheets (though honestly it's not too bad itself).

Using clasp, you can develop your code locally in an IDE of your choice, in typescript and have a build step compile those to js, and have clasp push it to spreadsheet.

Once you have the tool chain set up the DX is quite nice.

I spent some time with Apps Script a few weeks ago. It has some strange design decisions:

1) Everything runs on the server, including triggers and even custom functions! This means every script call requires a roundtrip, every cell using a custom function requires a roundtrip on each change, and it feels much slower than the rest of the UI.

2) You can't put a change trigger on a cell or subset of cells, only on the whole sheet. So you have to manually check which cell the trigger happened on.

3) Reading and writing cell values is so slow (can be a second or more per read or write) that the semi-official guidance is to do all reads in a bunch, then all writes in a bunch. And it's still slow then.

4) A lot of functionality, like adding custom menus, silently doesn't work on mobile. If your client wants to use Sheets on mobile, get ready to use silly workarounds, like using checkboxes as buttons to trigger scripts and hoping the user doesn't delete them.

Overall I got the feeling that Google never tried to "self host" any functionality of core Sheets using Apps Script. If they tried, it'd be much faster and more complete.

> 2) You can't put a change trigger on a cell or subset of cells, only on the whole sheet. So you have to manually check which cell the trigger happened on.

This is true of MS Excel's scripting language (VBA) as well. Worksheets are objects with events; cells are objects without (VBA-accessible) events.

It may be an issue with scaling and efficiency.

But Google Sheets remote procedure calls are vastly slower than local OLE/COM dispatching. (And VBA/Excel presumably uses the optimized tighter COM interface binding instead of the slower high level COM IDispatch. Sure there's some overhead but it's nothing compared to Google Sheet's network overhead.)

Not only is scripting Google Sheets indeterminently and syrupy slow, it also imposes an arbitrary limit on how long your code can run, making a lot of applications not just inefficient but impossible. Running your code in google's cloud doesn't make spreadsheet api calls any faster, it just limits how long you can run, them BAM!

To get anything non-trivial done, you have to use getSheetValues and ranges to read and write big batches of values as 2d arrays.

https://developers.google.com/apps-script/reference/spreadsh...

It's easier to just download the entire spreadsheet csv or layers and bang on that from whatever language you want, instead of trying to use google hosted spreadsheet scripts.

> Everything runs on the server, including triggers

I think that’s a consequence of the fact that multiple users can simultaneously edit a sheet. Yes, Google could special-case the “you are the single user of this sheet” case, but that’s extra work, and, I think, would be fairly complicated when handling edge cases where users frequently start and stop editing a sheet.

> I think that’s a consequence of the fact that multiple users can simultaneously edit a sheet.

No, it's not. Built-in functions like SUM recalculate instantly, and custom formatting rules (e.g. "color green if above zero") get applied instantly, even when there are multiple users editing a sheet. Running custom functions and triggers on the server is just a decision they made.

Sum and custom formatting are idempotent. It doesn’t matter whether that gets run once, twice, or a hundred times.

Arbitrary AppScript isn’t guaranteed to be idempotent. You have to run it only once.

This reason doesn't make much sense to me. Let's say I write a non-idempotent custom function. It makes the spreadsheet behave weirdly: recalculating a cell twice leads to a different effect than recalculating it once. Does it matter whether the function runs on the server or the client? No, the spreadsheet will behave weirdly in either case, even with just one user.

Can we make a programming language that will save developers from that? Maybe, but that would be very hard and that's not what Apps Script is trying to do. It already allows non-idempotence, trusting developers to write idempotent code when they need to. So it could run on the client just fine.

This is exactly the issue. I don't want server-side scripts, what I want with Sheets is the ability to run custom JS from the page itself.

Ah, thank you for that. I want to write an extremely basic transform on my spreadsheet, and Googling failed to turn up how one did that.

Or use the API to program in anything you want. We use Google Sheets for our accounting system, loading data via bank APIs and a cron-driven python script. We used to use Xero, but it couldn't handle the different tax regimes we operate in.

That looks like a great usecase. Would you be able to write about the architecture. A lot of us would love to be able to do things like this in Sheets, I'm personally trying to integrate a forecast estimate into Sheets

I wish there was a better way to make spreadsheets use a database as a backend. Most of what people do with a spreadsheet would be better done in a database - but a database needs a lot more training to use (if you don't have the training you will make something worse than spreadsheet with your database!)

Wasn't Microsoft Access basically that? There's also Google's AppSheet, which may be that too.

Microsoft Access requires more training (tables, foreign keys...) and from experience most of the people using Excel managed to bend it to do what they wanted.

The maintainability of the resulting systems was not great, but they did the job and worse is better I guess..

Add in to that: programming VBA for Excel spreadsheets is extremely trivial; programming in VBA for Access is second worst only to programming in Visio, MS Office's awful drawing program.

You know, I used Retool a lot, and it made me think that I'd actually really like a modern Microsoft Access.

Kind of an open source Google Forms/ Access where you could deploy front ends very quickly and have it hit a DB

Here's a modern Microsoft Access: https://visualdb.com/

Or try Airtable.

>Wasn't Microsoft Access basically that?

No because the datagrid in MS Access is too rigid and doesn't have the extensive slice-&-dice features of MS Excel. My first consulting gig was creating customized MS Access applications. Despite that experience, I use MS Excel today because I know that MS Access is too limiting.

It sounds as if Excel has grown to incorporate most database features.

Does it have atomic transactions yet? That's the main thing keeping me using small databases like Access even when a mere spreadsheet would do otherwise.

Have you tried Airtable? It works like a spreadsheet but is a database underneath.

>>Wasn't Microsoft Access basically that?

>No because the datagrid in MS Access is too rigid and doesn't have the extensive slice-&-dice features of MS Excel.

i'm not saying it worked or worked well, but i'm pretty sure the point of Access in the office suite was so that you could access Access (get the clever marketing?) data from within Excel and then do all the excel things you were used to.

anyone know if that worked or didn't? DDE and all those other projects were always pursuing this as a dream

I think it worked technically and I‘ve seen a few Access-based solutions to problems faced by people/groups without any real access to development resources. However, these solutions pretty much always came from people with rather technical mindsets and I think many of them involved tinkering in the evening and on weekends. So, my assumption would be that the demise of Access was more about UI/UX, complexity, familiarity and the like than it was about functionality.

Access's database is fairly limited and prone to corruption, especially using in a (local) network setup. The better solution would be to have real backend database and use ODBC to sync in data to Excel and Access. Maybe back in 1995 it made more sense but that's before my time.

Access was pretty amazing on its own back in its day, ignoring its multi-user limitations. It glued together a relational database, visual query builder, GUI/Form Builder, and reporting. You could create forms with sub forms that linked tables together. Also had a datasheet view. All of this without touching VBA code, but VBA was there when you needed it.

dbase II offered similar capabilities on 8 bit CPM workstation in the early 80s.

> I wish there was a better way to make spreadsheets use a database as a backend.

Here you go: https://visualdb.com/

Years ago I used Excel (VBA) as a CRUD interface on a database with minimal validation, it didn't take too long.

Pretty sure something fairly similar is built-in these days.

Hard agree here. The value of combining database + UX + logic into a workbench is like the app we've been reinventing over and over again. It's why Visual Basic still lives :-).

And yes, not the greatest way to proceed once you know what you want, but a heck of a way to iterate quickly and identify the actual requirements.

The Google suite of Apps was born multiuser. They perform better then Microsoft's apps when multiple people, even dozens of them, are bashing on a single shared document. Document sharing is more powerful than sharing a window or a desktop. It changes the way people work together.

yes, that's true. It also brings the issues related to that - it's easy to delete the whole org data, change things uncontrollably, corrupt the data, there entries in the tables are free-form (well, you have to do programming to make it more robust).

it works fine initially, but for long term it's usually more productive to have custom software.

> it's usually more productive to have custom software.

If you multiplied the custom software development effort and cost to the multitude of places and use cases people in an organization are using spreadsheets I think you’d quickly find that it’s infeasible. And that’s not even taking into account how much value the adhoc-ability of spreadsheets adds to the equation. Most spreadsheets can be completely refactored or thrown away in a rather trivial manner. The sprint nature of software development screens out most things that could be spreadsheets. I’ll build it in an hour instead of waiting 2 weeks to get on the next sprint.

The software development process is too rigid for rapidly changing business needs. Having to spec out requirements and such is often an unknown and something you’re doing live in the moment when creating the spreadsheet itself.

Google Workspace apps all (?) have versioning.

This is true, but sadly for very large spreadsheets, it kind of stops working at a certain point. You get timeouts when loading cell history or the version of the spreadsheet version history.

I would not have agreed readily with that statement until three years ago, when I actually started working in GS as part of my job. I couldn't have imagined how seemless and well-implemented collaborative editing had become IRL.

>>It's the best authoring tool we've ever devised.

100% agreed. Creating a spreadsheet is declarative programming, and Excel (and now Google Sheets) has made more developers than any other platform (probably by an order or two of magnitude).

I do not know a business that was not CRITICALLY dependent on Excel for actual business operations through the 90s and 00s...and the same is likely true today.

I've long wondered why there's not a more incremental transition path from spreadsheets to a full blown database with a custom UI. It just seems like spreadsheets are right on the cusp of being able to fill that that role but lack just a few key features which might make it viable (structured data, native SQL query support, custom UI elements, IDE integration, etc).

> incremental transition path from spreadsheets to a full blown database

There are plenty of tools for that. Airtable comes to mind, or if you want to use your own database try: https://visualdb.com/

At my old gig, all of our content generation was in Google sheets or Airtable.

Anything we made would have _never_ been as good a UI and would have stopped us from being able to publish new content. The other nice thing about them is you can scale the integration effort. You can download a sheet from GS as a CSV then upload that in an admin page and you can probably ship that feature today. A tiny bit more effort and you can add a 3rd dimension to the data by finding an XSLX library and exporting the page as an Excel workbook. Eventually you can do something with their API where you just click "reload data from the master sheet" and you can generate all the data you need to preview the changes on the frontend. That's what our Airtable integration looked like. Airtable was very "pull this in via API" native and had a better way of expressing higher dimensional data in cells which matched nicely with Postgres storing arrays in cells.

Btw, anyone ever used Mathesar ? I saw it on the hn frontpage but I haven't tried it yet

I didn't realize it at the time, but I my first encounter with the "solves every problem with a spreadsheet" user type came very early in my career.

I worked in a computer lab in college, ca. 1989. One of my colleagues was in the mechanical engineering program, and had a bias generally for "solve the problem" over "elegant solution" or "appropriate tool" concerns. (I love the guy to this day, to be clear.)

When he first came to work at the lab, he was the only guy to have installed FORTRAN on his workstation. It didn't work well.

Then he discovered Lotus 1-2-3 and its macro language. He DELIGHTED in making the rest of horrified by creating all sorts of boundary-pushing utilities in Lotus macros. To be clear, he was at least 50% "doing a bit", and leaning into the "engineer only knows one tool" gag we'd all been riffing on. But he was still doing absurd stuff in Lotus that would've been better built in, say, Turbo Pascal or Turbo C.

I had no idea back then that this pattern would become so prevalent.

I worked at a law firm where the chief accountant (not the accounting manager, but the guy who made most of it work) did everything in Excel...including writing memos and other documents. I think the only other program he used was Outlook, and he probably drafted email in Excel.

  > especially so for people who can't code.

Presumably those people learned to use a spreadsheet. What makes learning spreadsheet formulae possible, but SQL, Python, or R impossible?

nothing makes it impossible, but they already have Excel installed on their computer and they learned how to use it. Depending on where they work they may not be able to install Python or R or a database.

SQL and to an extent R were both created under the assumption that most people were not inclined to learn to program the computer. They could be expected to speak English and understand relational algebra, but they couldn't be expected to learn an imperative programming language. The existence of SQL stands as proof that at least some people 50 years ago thought there were multiple classes of user.

It is far easier to work your way into programming via spreadsheets than via sql.

One day you are writing ‘sum(a2:a201)’ the next you do some conditional formatting and so the complexity builds up very slowly with your needs.

With sql, day one:

‘SELECT SUM(Column) AS Total FROM Table;‘

Way more complicated. Way more powerful too, but most people don’t need the power until much later.

And you have to work in the console which is an unfamiliar ui for many.

It’s not impossible, but very high overhead in comparison.

That's not true, you can use Airtable to get the ease of use of a spreadsheet with the robustness of a database.

And now you have two problems. It’s not just SQL that you have to learn, but airtable. Which probably isn’t installed anywhere.

Not true. You can't use SQL in Airtable even if you want to, it just doesn't support it. Also, Airtable is SaaS, you can't install it.

s/install/somehow obtain access/

Doesn’t really change the point.

As of this writing, there are twenty-two comments on this thread about airtable. All but four are either you or responding to you bringing it up.

I think it’s fair to say that whatever its advantages may be, it is relatively unknown, and therefore a higher-overhead entree into programming than straight up spreadsheets.

Just learning about it or finding fit for purpose takes more time than diving into a spreadsheet. And as always the argument is not that spreadsheets are more powerful or better, but that they are much easier and more incremental to learn and start with.

You are absolutely right and it makes sense to start with a spreadsheet. As I said elsewhere, you have to reach a pain threshold in order to look for alternatives to Excel. And how soon you reach the pain threshold depends on how big your data is and how many users are trying to modify the same data at the same time.

Google Sheets has the =QUERY() function which does some very limited SQL.

Spreadsheet formulas have a very simple grammar: expression = reference | exp1 op exp2 | function(exp1, ..., expN)

That's it. It's the math notation from high school, plus cell references, which can be inserted by clicking a cell.

Telling everyone to learn Python to solve all of their problems is not a legitimate or realistic suggestion or solution.

Why? We expect huge swaths of non-programmers to learn R to work with data. Seems like it just depends on the field. If Excel was an integrated Python environment (or VBA or whatever pick your language) then I bet users would have learned it all the same. Twenty years later I bet HN would be full of complaints about how people have a hard time transitioning to "real" Python because they learned the Excel flavor.

> We expect huge swaths of non-programmers to learn R to work with data.

I’ve worked in academics and industry around biologists, chemists, physicists, statisticians, bioinformaticians, and all varieties of engineers.

I’ve never seen “huge swaths” of anyone expected to learn R for anything outside of a few niche areas in statistics and bioinformatics.

What people are expected to know is how to use a spreadsheet. What people are often given is Microsoft Excel, and essentially nothing else. A lot of companies wouldn’t dream of letting random employees install or use R or Python.

It’s not ideal. But some battles can’t be won and aren’t worth fighting. Which is why people use Excel for so many things.

Do we?

I never seen R being used, when Excel doesn't work out, it is something like Tableau instead.

Because they don't know it's programming. Seriously. I dated a woman for five years who was an accountant and a spreadsheet wizard. I told her that doing what she did with Excel, she was just as much a programmer as me, and she flat-out refused to believe that she had the mental capabilities required to be a programmer.

Years after we broke up, a new company she joined required her to take a VBA training course, and she texted me and told me I was right, VBA was easy and her biggest challenge was being patient while the other students struggled.

Discoverability is a big one. People really underestimate the value of having a menu, an actual fucking menu, listing all the functions you can use. Typing and getting autocomplete is a poor substitute, and you don't get even that in SQL, Python, or R by default - there are products that can provide that for you, but only after an expert has set them up.

Most people who have access to a spreadsheet don't have access to a compiler or interpreter, or the desire or time to learn them if they do. Most people learn what they need to do their jobs, and then go home and forget about it until the next day.

I agree. However, so many of my use cases include a one-to-many relationship that I was outgrowing excel/sheets too quickly. Once a project added a VLOOKUP, it hit an inflection point in complexity.

I spun up a local Grist instance in my org, using SAML with our org's email authentication. It's intuitive enough that I've replaced a few shared spreadsheets with it (now with rowwise permissions) and powerful enough that I've also replaced a few internal CRUD apps.

https://github.com/gristlabs/grist-core

While Grist is a brilliant product, I really dislike the spreadsheet marketing they use.

It's not in any way a spreadsheet, you can't use rows correctly and everything is stuck in columns. Not to mention the forms you use for input are not user friendly compared to a simple understandable coloured cell

I agree spreadsheets are amazing, but they are not databases. Much grief has been caused by trying to make them be one.

Have you not heard of spreadsheet-database hybrids? Like Airtable? Or if you want to use Postgres: https://visualdb.com

The gap in conceptual complexity to switch to databases is just way too high for most people. One lets them accomplish the business goal, the other does not.

And yet...they are often used as databases, successfully.

>> especially so for people who can't code

Minor point, but I would say that people who can construct formulas, are indeed coding.

To expand on the overlooked point: it gives you a DB and a programming environment (however challenged) that you can use without needing sign-off from IT. In any moderately sizeable organization, getting approval to use anything but standard software is slow and painful.

Nobody wants to explain to IT that they need to install Python on their machine, or drivers for sqlite, or - god forbid - get a proper database. Because that requires sign-off from several people, a proper justification, and so on.

> RIP HyperCard, may you be never forgotten.

There's Decker https://beyondloom.com/decker/

> It's the best authoring tool we've ever devised.

It is also a very bad co-authoring tool imo unless you have a very tight team and processes of how to use spreadsheets. I would not mind if I was using it alone. But spreadsheets encourage a certain naive "visual" approach to structuring data in them that can become an issue if you want to import the data to actually process it somewhere and your coworkers don't really understand this well.

Spreadsheets can be useful, however:

1. Speaking about excel in particular, localisation issues are an absolute nightmare if you happen to live in the wrong country (commas vs dots for decimal points) especially while importing CSV files (which is already unnecessarily complicated in excel). If somehow you don't notice these issues, you end up with wrong data without understanding there the error came from.

2. If your coworkers do not really understand very well how spreadsheets work, you quickly get to become really frustrated with issues coming up when you have to import a spreadsheet for actual processing. Datetimes with different forms, coloured boxes being meaningful, mixtures of text and numbers and whatnot.

Yes a big part of it is "people problem" rather than "technology problem" but imo spreadsheet technology (excel, google sheets etc) encourages a variety of practices that make it less reliable. If the technology gives you the freedom to mess up too easily, imo it is not just a people problem.

Having worked on Lotus Notes, I am acutely aware that groupware wasn't the answer either.

I agree. It's extremely hard to learn to use someone else's spreadsheet unless they're sitting right there with you watching you fumble through it.

I have a couple of spreadsheets that I use on a weekly basis, and I found it easiest to build both of them from scratch, despite the tons of examples floating around for exactly my use cases, and despite it being my first time building anything with a spreadsheet in 10+ years.

On the bright side, after using these spreadsheets for a week, I lost all desire to write apps to do the same things. Google Sheets is a good-enough UI and solves sync across different computers and mobile.

i never loved nor hated hypercard. it was straightforward and clever but i really only used it for one thing: i made cardstock labels and liner notes for my audio cassette "rips" of albums (my 20mb hard disk (we didn't have MiBs then) wasn't going to hold many mp3's ;)

i might have used it for burned CDs too, I can't remember. at work i had access to burning CDs at scale back then, I just don't remember when the blanks became cheap enough that nobody would notice my pilfering and and whether i still used my mac. i did burn disk backups to CDs

I would agree with you except the UI is so horrible.

Give me a spreadsheet with a world cksss user interface and then you’ll have something!

Spreadsheet is the user interface.

Also what you're looking for is called Microsoft Excel. Every one else is doing webshit UIs which can't be world-class by definition, because they hardly scale to real workloads :).

And not to forget, you can run Doom and Tetris in them.

In both GS and XL, actually.

Excel used to have its own video game as an Easter egg.

> A spreadsheet gives you a DB

They can emulate behaviour of databases; but the missing parts missing will haunt you. Spreadsheets are a jack of all trade, mastering nothing, haunting you with everything;.an amplifier for the Dunning–Kruger, where people are misguided about data-quality.

Spreadsheets are indeed a great tool, but the implementations we have today are bad, with too many booby traps, not enough safeguards, not even much comfort for those with higher demands.

> spreadsheet gives you a DB

Yeah a DB where any user can accidentally hit the spacebar and erase a formula, and never see any warning that their outputs are now horribly inaccurate.

Data > Protect sheets and ranges, choose Protect range or Protect sheet, and then click Set permissions to customize who can edit them.

What Problem?

and they save versions every edit or every few minutes too. It's free version control that you cannot forget to turn on

Huge contrast with MS approach to this

MS approach is the same these days actually, people even complain about it.

When was last time you used Onedrive or Sharepoint connected Excel? That's also how it works nowadays.

I am anal about doing this at the range level, but few people are.

Another oft overlooked point is that anti-spreadsheet people aren't actually against using spreadsheets, we just want our coworkers to stop placing mission critical operations in an enormous, half-baked spreadsheet that exists only in some long-since retired users home directory.

I see these complaints on HN a lot, and maybe it’s anecdotal, but I just don’t see this in the real world these days.

If someone shares a sheet with me, it’s for the intended purpose of sharing data and/or visualizations of that data.

I’ve always been a huge fan of spreadsheets, and the rare times I’ve encountered them being misused, it was a long time ago, and not near enough to make me an “anti-spreadsheet” person.

It sounds like these anti-spreadsheet people need to find a new place to work and/or new coworkers.

Either way people shouldn’t be anti-spreadsheets because some people misuse them. That doesn’t change the fact that they’re a great tool for tracking/sharing/visualizing data.

This is anecdotal bias even now. The number of these monster spreadsheets running organizations that should be more sophisticated than they are would most likely keep anyone here up at night.

> I see these complaints on HN a lot, and maybe it’s anecdotal, but I just don’t see this in the real world these days.

It happens all the time where I work. I don't want to be specific, but we have lots of examples here. In some cases people don't like the core software, so they work around it by tracking things on a spreadsheet. And sometimes that spreadsheet disappears (in one case, it was being kept on an XLSX on a USB thumb drive, but the thumb drive got corrupted and we lost some very important data.)

The availability angle changes things quite a bit. Having a single source of truth online sheet is much different than a file that is passed around.

This depends a lot on what you do. Try working with a decision analyst sometime. The entire economic model with a decision tree and monte carlo analysis of cost overruns, etc for a multi-trillion dollar decision will literally be a arcanely-complex spreadsheet or two on someone's laptop.

With that said, it's still a great tool for the job because the different stakeholders can inspect it.

>I just don’t see this in the real world these days

Lol. Go to literally any bank. They all have a legion of accountant,analyists that's sole job is to maintain their little fiefdom of spreadsheets that only they understand.

If most people knew just how held together by string,tape and gum the banking industry is there would be a run on the banks.

There is also always some 75+yo part time guy that maintains some sort of critical system. He always says, "I want to retire but they keep throwing more money at me"

I was part of a team helping a Fortune 500 company whose inventory system was basically a master spreadsheet that maybe 8 people in the company actually had write access to. Everyone reported numbers up to them and the spreadsheet had read only views for purchase order projections.

To say that it was a nightmare was an understatement. They were willing to dump vast sums of money to get something better, but they'd homebrewed so many human processes to deal with the spreadsheet that they struggled to adapt to a more conventional way of doing things.

Found the GAP employee.

And if you aren't this story could have come from one.

Not GAP, not even clothing, but still retail.

My big gripe with spreadsheets is just that there's a lot of bloat on top of the spreadsheet which makes large datasets difficult to work with.

Otherwise, I find them to be great deliverables.

How large are you talking about? Have you tried https://rowzero.io?

>but I just don’t see this in the real world these days

May be due to the fact you're a single datapoint and not omnipresent at every company. What a weird argument.

As opposed to placing the mission critical operations in some huge enterprise Java deployment which has been touched by thousands of contractors, all of whom have only a very rudimentary understanding of the actual business logic.

I think even a complicated spreadsheet that can be directly edited and modified by the actual business stakeholders is preferred.

> even a complicated spreadsheet that can be directly edited and modified by the actual business stakeholders

You can do that in a spreadsheet-database hybrid such as Airtable.

The main problem is that a complex spreadsheet is just code. Its a bundle of logic and data. That's code.

If the business stakeholders can edit said spreadsheet, they can code. Not well probably, but they can.

So, theoretically, they should be able to open a python script or whatever and hack away. A lot of calculations are actually much easier and straightforward in a real language as opposed to Excel.

But they won't, partially because developers would never let them.

You need to give them a tool that's as easy to use as a spreadsheet, and yet stores data in a relational database. There are plenty of spreadsheet-database hybrids in the market.

Are any of them as easy to use as a spreadsheet?

I would say Airtable is. Or if your users are slightly more technical you can try https://visualdb.com/ because it lets you use your own Postgres instance as the backend db.

I’ve used Airtable a bit. I think it’s really cool and would like more people to use it. However, it’s a lot more clicks and key presses to get things done - especially data entry - than in Excel. This also makes it better, since you can put constraints on tables, for instance.

You are absolutely right about needing more clicks and key strokes in Airtable. You have to reach the pain threshold in order to look for alternatives to Excel. And how soon you reach the pain threshold depends on how big your data is and how many users are trying to modify the same data at the same time.

See here to understand what you are missing out by not using a database: https://visualdb.com/comparison/#integrity

I feel quite comfortable in Excel - used various tools like Power Pivot, Power Query, OLEDB, created my own functions, Python within Excel, etc. - but Airtable felt so confusing and limiting. Other former colleagues also struggled with Airtable; maybe it was not explained to us correctly.

NocoDB is also a self-hosted option. I used it to power a family project that required regular API access, but without the rate limits of Google Sheets.

Using any tool but Excel has the same problem as finding programmers who are willing to use some obscure proprietary programming language. They're going to worry about career lock if they're not developing portable skills.

Even though Excel is proprietary too, it's ubiquitous enough that people don't have to worry about it.

That's true, nothing is as ubiquitous as Excel. But Excel is not designed for multiple people simultaneously updating data. At some point you need a database.

It's not perfect, but Excel can track changes by multiple users. Excel files shared on OneDrive/SharePoint allow multiple people to simultaneously update the data, and it tracks each individual change by each user.

What about a tool that tracks updates to shared Excel spreadsheet and replicate them in a SQL database?

And then, if somebody makes a change in the database, a trigger will update the spreadsheet

Such a two-way binding makes it possible to continue relying on spreadsheets for UX, all the while the data is not locked in there and we can also have other processes handling the data (a web app, some cron job, etc)

Maybe market it as an API for excel or something

I don't use Excel much, but I do the analogous kinds of things with Python and Jupyter notebooks. The problem is that adding some kind of layer requires the involvement of corporate IT, which blows away the advantages of DIY'ing things.

As someone who has been to multiple trade shows to show off our own spreadsheet product that solves some of these problems (https://rowzero.io/home), I can tell you there are a bunch of data engineers and their managers who have a visceral hatred of spreadsheets but have trouble articulating any reason for it.

Feels a bit like the no-code paradox though. When you make a sufficiently advanced no-code (or no-database) tool it becomes equivalent to code (or a database) and even though it does entice a novice to start using it, a complex system made in it is still complex code (or a complex database) and ends up requiring a competent programmer (or db admin) to maintain it, who wishes it was written the normal way to begin with.

I have no doubt that the stories of complex spreadsheet models as the beating heart of a business are true. But those models are likely the sole survivor from a thousand dead and gone spreadsheets that people extracted value from for a time, but just didn't have the ongoing utility or wide enough audience to merit being turned into a dashboard or web app. It would be a mistake to insist that all of those spreadsheets should have started life as something else, just in case maintenance should someday become necessary.

In other words, one of the core use cases for a spreadsheet is that it empowers a broad swathe of users (broader than Tableau or PowerBI) to quickly extract insights from their data to fill immediate needs.

Or at least, that's a core use case if you can get your data into a spreadsheet without too much trouble.

And then the attached part of this that you mentioned is that the cursed survivors have essentially outcompeted their peers for survival. They're winners of a genetic algorithm whose basis is company data. They have close contact with actual reality, and have been beaten carefully into shape by it. I have kind of a grudging respect for these because of that, actually.

> It would be a mistake to insist that all of those spreadsheets should have started life as something else, just in case maintenance should someday become necessary.

The problem corporate IT/Dev folks face isn't that an idea started as a low-code tool, but rather that the low-code solution is often dumped on them with no budget or desire to improve it to be more reliable and maintainable.

At least until something fails... and usually in dramatic fashion that then wakes leadership up to the idea that maybe we should invest more into this critical business process. If the company didn't go under in the meantime.

I love spreadsheets for casual stuff, but my concerns with using them for anything heavy duty are twofold: change management and correctness testing.

Other software that I use and write is version controlled and has tests to catch errors, mistakes, typos. Those tests regularly find and prevent problems! Likewise version control.

Could we get the same with spreadsheets? Seems difficult but not conceptually impossible, particularly with LLM assistance.

Imagine if people used Emacs the way they use Spreadsheets. "Hey I made a way to simplify our workflow, it has forms, buttons, gizmos we need" then you go to their desk, and its some custom Elisp abomination. I have to imagine this is how most people who hate spreadsheets see them. It still works, but it's a quirky solution.

Also I remember Row Zero the demo on your home page was impressive. You guys were S3 engineers too, good to know your project took off. :)

> but have trouble articulating

I imagine that's because it's not really a technical problem, but an issue with how the whole organization (mis)handles complexity, and we collectively [1] still struggle to model/name a lot of those problems.

[1] Yeah yeah, I see you there in the back, you excited cyberneticist bubbling with enthusiasm to share... but I mean as a practical widespread matter.

I've always suspected if someone were to make a spreadsheet app that solves most spreadsheet problems it would collapse most companies.

As I see it the limitations of spreadsheets are what keep them from becoming the official way of doing things. At some point most of the stuff in the spreadsheets has data provenance in a properly managed IT system somewhere. If companies could get rid of this part of IT they would suddenly find themselves in a trillion dollar pit of lost money/inventory, As the spreadsheet mess spiraled out of control with no source of truth.

One of the surprising things about working inside of $MegaCorp is that if you knock on enough doors, you'll eventually find that each org has, like, one dude* with a spread sheet that powers everything else. Teams will get spun up to try to "automate" this spreadsheet, but, on a long enough time horizon, the spreadsheet wins.

And it was probably created without approval.

... and been changed 3 times in the weeks it took to deliver the app.

Similar to the author’s cases: I’ve sat through several presentations of small teams who worked for 6-10 weeks to create a web app that does something (the details vary slightly from case to case) where at the end of the presentation, my first thought is “what a waste of effort and money; that would have taken 2-4 hours if they’d just used a spreadsheet”.

This speaks more to the complexity of web and web development than as a pro of spreadsheets.

You can create a super quick Python app with visualization and whatnot in about the same time as a spreadsheet. But then it's not online. And it's not best practice.

Web is just a fucking beast and then developers go in and add additional complexity.

It would probably be a much simpler web app if you made it just, like, a bunch of PHP scripts thrown in a folder. But they won't do that.

Excel, creating job security since 1997.

That's every company I've ever worked for, whether they have 100 employees or 300,000.

so true - as the spreadsheet "guru" at most of my employers - most of the issues with spreadsheets is misuse which is very common sadly

though, much programming is poor and often can be accomplished better in a spreadsheet given the situation and use case...

Poe's law. If you use a spreadsheet as a DB for any sort of application, you should be tarred and feathered, and never infect any engineering team ever.

Your 'trifles' are the biggest screw-ups from functionality to INFOSEC.

MS Excel is much much much much more accessible to the normal office worker than MS Access. It's not even close. As a kid, most of us intuitively picked up Excel but Access was a mystery even with a teacher in computer class trying to teach us the basics. Later I learned SQL to create websites, and graduated with a good CS degree, but the learning curve is uncomparable.

Reply on news.ycombinator.com