Hacker News Reader

On top of what you're saying, the alternative is frequently a heap of under specified, under tested (and only manually tested!), copy-pasted and shoddy quality Cobol/Java/Oracle/C#/SQLServer/PHP/MySQL code.

Yes, the performance will generally be better (1 minute to run a report, not 1 hour), but I'm not sure things will generally be better. As you said, the business people can at least modify the Excel. These systems? Not so much, without an usually massive support contract (Yaaaay! for us IT people, I guess).

People sometimes make a false equivalent and imagine that the Excel jungle will be replaced by a Google-developed, 90%+ unit tested, refactored and cleaned periodically bunch of code.

> the business people can at least modify the Excel

I'm not convinced that's a good thing. If they can change the Excel then they can also fuck up the system, and if you fuck up a system like this then it's horrendously difficult to unfuck. I think systems like this look great when they're working, but the second they go tits up you realise why people are horrified by them.

Users able to modify cod is the opensource model. It doesn't lose its benefits and advantages just because the users are not us(professional devs)

It's not so much being able to view or modify code that's the problem, but the fact that any modifications to an excel spreadsheet are inherently running in production. Open source has many benefits, but while open source projects may accept pull requests there is very few which put contributions straight into the latest build without review.

That's not a realistic comparison. I have yet to see one user try to modify Firefox's code. Spreadsheets, on the other hand ...

You are still making a distinction between user and developer. I think the point is that we can all be both.

That's not the issue at all. The issue is that these systems invariably have no review process, no testing, and no versioning (at best they'll have each spreadsheet versioned with no real way to know what each version contains). And this makes them incredibly fragile.

Especially since the people who do this generally don't use version control.

They do use version control. They put each version in a folder named after the date

And these user-developed systems are often great...until someone accidently deletes or changes some data and it becomes a mess.

Unfortunately, many businesses then find they have no way to know the extent of the data lost or who did it. And there’s no proper backups. Happens over and over thoughout the world.

Excel’s a great tool and there are lots of terrific uses for it. But what the article describes isn’t one of them. If we’re being honest, this isn’t the first time the issue’s been discussed and the companies likely aren’t actually switching away from excel, but rather supplementing it.

In fact, the article seems to be more a “hit job” / “marketing piece” for Anaplan Inc., Workiva Inc. and Adaptive Insights.

I agree, the issue is not Excel as such but how it's being handled: I once wrote a blog post on how you can unit test Excel spreadsheets, too: https://www.zoomeranalytics.com/blog/unittests-for-microsoft...

> The alternative is often rough. Software projects like this have high failure rates. Getting “requirements” out of users’/clients’ heads and into programmer’s is difficult. It means the standard (natural?) gap between domain knowledge and programming knowledge is restored. That has costs.

The alternative doesn't have to have a high failure rate. I was brought in to help reboot their in-house implementation of a product data management system - the core issue was they didn't have any real competency in software development (though they believed they did.) Ultimately I recommended they abandon the in-house effort and instead proposed using an off-the-shelf PDM system, we evaluated quite a few decent options over the course of six months and found several that met the business's requirements. It would have been the right thing to do and I believe it would have been successful. Culturally the IT department staff was plagued with not-invented-here syndrome. It was a generally dysfunctional IT group and it's quite likely that Excel got it's start as the system of record because the business needed to get the job done, and the IT department was unable or unwilling to deliver.

Alternatively you misunderstood the business requirements and would have cost then a fortune on a failed project.

Alternatively you were brought in by management to make cost cutting recommendations - and your solution wasn't cheap enough.

Alternatively the company accounted for the enormous risk of the project succeeding but actually being worse than what they had in place.

You really don't know the politics of this.

Edit: Another alternative is that your plan would have resulted in sufficient job losses that the company was worried that it would demotivate other employees and damage the company more than the cost savings.

I upvoted your post because I think those alternatives are interesting to consider, but _you_ know far less about the politics of the situation than the OP.

It's entirely possible that the OP was exactly correct, or close to correct. It's also quite likely that at least one of the factors you mentioned (or others) was also at play.

OP here - the director that brought me in was (and still is) a straight shooter. There's no question as to why I was brought in. While she had the support of the CIO, the rest of the directors spent more time protecting their kingdoms than trying to modernize their systems. Fortunately for both myself and the director, we moved on from that organization.

Make it easy to interoperate with databases and/or revision control painlessly for the end user and the developer. The users are fine with excel, it's the attendant issues like centralizing data, running automated services off of the data, backing it up, and keeping an audit trail that created pain when I worked in healthcare.

If excel online was better, a lot of that would be pretty doable.good points.

But..I do think that if we're thinking in terms of "users" and "developers" then we're no longer talking about excel. The whole reason excel is used this way is the continuous spectrum of sophistication.

Also because Excel is both an IDE for developing spreadsheets, and a GUI container for operating them. Splitting dev time and run time environments would help with control. When a dev supplies a user with a traditional .Net GUI app, they don't enable the user to hack the app by supplying a copy of VisualStudio alongside. Effectively, this is what Excel does.

Yes, but.

It's no good if you fix excel by making it not excel. The whole reason excel is so popular is the intuitive, unintimidating but terribly wrong way it mixes data, code and GUI. Functions in the data, constants repeated in every row, hard coded variables all over the place. If you're just supplying a GUI and hiding the code, the user can't change, add or debug anymore. The point is that they can code, even if they don't know they are coding.

The trick is to think up a way of shipping VisualStudio, but making it less terrifying to engineers, managers and such.

Agreed. But a different way of working the same trick is to have an alternate run time that can execute .xls[x|m] as a headless server process.

"If the excel team was blessed with a plague of inspiration tomorrow morning, Is there a way they could make it so that excel is the right way to manage an insurance business?"

What a professional software developer's workflow has and Excel lacks:

* principled way of sharing code across a team with distributed source control (or similarly powerful tool),

* automated regression tests,

* input validation,

* continuous deployment tools to automatically deploy working code (default mode for Excel I guess), and a mechanism to roll back to a previous version (not so much),

* build and dependency management tools for bringing in libraries from a vast open source ecosystem,

* systems for reporting bugs and feature requests and tracking their progress,

* networked database that is the shared "source of truth" for all users,

* security and auditing,

* hardware capacity planning (run locally or in AWS? what are the cost trade offs?)

So sure, get the health insurance people doing all of those things, and have Microsoft figure out how to do all of them well in Excel, and congratulations, they are now software developers. None of these require using a "real programming language", at least not directly.

That's great (though I'd quibble about a few of them, eg I was doing input validation in Excel 25 years ago, and boy do I feel old now), but while you're learning to do all that great stuff you're not learning anything about the domain you want to work in.

OK, I don't know who wants to work in health insurance but the basic point is Excel is a good tool for people who have specialist knowledge in a particular domain and want to automate their work. Programmers know the best way of programming but frequently overestimate their understanding of other people's problems.

You underestimate how difficult it is to work with at least some software companies, how expensive it is, how difficult negotiation it takes to convince quite a few nerds that user actually has business case for that requirement. And many software companies produce poor quality and are very slower then necessary despite using all buzzwords and all nighters. Managing even outsourced projects takes learning and expensive failures are nor an exception. All in all, the software we all collectively produce is still very expensive and unreliable.

Even absent that, it is difficult to explain what users need to coders who have zero idea about the domain. Unless it turns out they are the kind that just know user is idiot the moment she has different opinion on her own job, in which case expect needed time to tripple.

The homegrown excel is awesome analysis once the company decides to move on. It is concrete, you can generate test cases from it, it has features.

Integrated version control is one of the biggest things I wish they included in Excel rather than relying on an external system to do it.

One other point is let's say you do create some fancy new system to handle the task. It's likely going to erode and be surpassed by new computing technology at which point you're looking at an expensive legacy migration which could prove to be costly and you have the same downsides you mentioned.

Excel upgrades don't seem to break previous files as severely and have the backing of Microsoft.

I can't believe I'm defending Excel.

>If the excel team was blessed with a plague of inspiration tomorrow morning, Is there a way they could make it so that excel is the right way to manage an insurance business?

No... but I think the converse might be possible. If those with the programming knowledge sought to replace the Excel system but in a way which actually preserved the useful features of the Excel solution. Dirty, filthy things that programmers don't like. No referential integrity, data duplication, deferring decisions to human beings rather than lists of rigid criteria, etc. If they could swallow their bile for just a bit, they could build a system just as useful and flexible and introduce tools to add in the things like data consistency and referential integrity - tools that the user would initiate, thus accepting responsibility for the restrictions such things impose on themselves and the data.

That is not, unfortunately, how it is usually done. (I am currently literally working on a program to replace a 40K row Excel sheet that crashes multiple times a day which has a workflow centered around it that involves slicing and dicing the sheet, passing it off to multiple different groups for separate editing, etc... not sure I will be able to sell the team on this approach, however I do believe it is sound.)

| I am saying that we should be curious at why they are doing it this way.

If you hand a caveman a machine gun, he will club you to death with it. IOW, the casual ability for domain experts to screw up a spreadsheet is enough of a reason to decide that excel is a terrible solution to this problem.

I would make a different caveman analogy. Hand a caveman a gun. He probably won't be a better rifleman than you (or someone). Really bad form and technique. But, he'll probably be a way better hunter than you, shoddy rifle technique or not.

Caveman were better than you at a lot of things. They knew how to find squirrels