Hacker News Reader

Comment by rajacombinator | Hacker News Reader

Pretty mind boggling. How does one spend $1 billion on software and fail? I guess this is a somewhat misleading accounting statement based around maintaining the existing software orgs of the merged companies, not building new software.

It makes me wonder though, with an unlimited budget and willingness to pay a lot, what could one achieve in one year? If you offered 200k base, 100k signing, and 1 mil bonus for completion of goals within 1 year, that should give you access to pretty much all the top tier programmers in the world. Hire 50 of those guys and you're only at 65 mil. Maybe you need a really incredible guy running it so you offer him a 30 mil bonus. Now you let those guys attack the problem and it's hard to imagine they couldn't fix things up at DHL within 1 year.

Any known examples of orgs trying this approach?

View on HN

"How does one spend $1 billion on software and fail?"

Just look at most UK Government IT projects. They've got it down to a fine art, blowing billions and being left with nothing useful.

Several major international government consultancy firms got their break on supposed small UK Gov IT projects, and were able to keep billing $1,000's/hour for years (while delivering nothing useful) and became big IT consultancies in the process. The failure didn't harm them one little bit - it let them claim the have experience and that got them way more work.

IT project failure in large and complex organizations is practically guaranteed.

Now, if they did that small-ish team, paid well, it could well work. But your board is not going to approve some pizza munching, jolt cola drinking geeks as your wonder IT team. Don't worry what the team is really like, the consultants asked by the board to review them as part of the process will make sure they're described as the aforementioned pizza and cola weirdos. And recommend using the consultants team instead. Been there. Done that.

There's no board for UK government IT projects.

The process starts with lobbyists (from the likes of Fujitsu, CSC etc) offering fancy dinners, a few trips to the opera, family holiday in Barbados, and crucially a revolving door, behind which is a future lucrative senior position should they decide to engage the services of said company. Lucrative senior position will in part involve lobbying former government friends and colleagues.

And so the cycle continues...

NHS Connecting for Health literally was that board.

Also, never attribute to malice that which is adequately explained by stupidity -- or, in this case, incompetence. If you're telling yourself that the difference between billion-pound boondoggles and small, competent and motivated teams working to agile principles is a bit of low-level corruption (so lightweight that none of it could be discovered and prosecuted under the Bribery Act), you're looking in the wrong place (and you'll also struggle to explain how and why the Government Digital Service was set up, and why they're succeeding).

Yeah, you're right. I am generalising and cynically caricaturing; though a fair bit of that stuff does go on. The current system of self-regulation is sorely in need of reform and legislation, but those tasked with legislating are the ones who benefit from the current system.

I have a lot of respect for the lads and ladies at the GDS. They're doing some sterling work.

The NHS blew £10bn on one software project that never delivered and was scrapped. This is very normal in UK govt procurement. See also what we are spending on aircraft carriers with no planes.

Don't forget the ongoing costs. Nhs choices costs £3 per page-view from last year's figures. It's a largely static site.

Imagine, they could actually pay health care workers properly if they stopped hosing taxpayer money down the gigantic nepotism drain.

It's horribly slow and doesn't seem to work well at all neither. Hopefully the new NHS digital team will take over that entirely soon (http://nhsalpha.herokuapp.com/)

The sad thing is if you say this is evidence the NHS needs some reform, a load of freaks will appear screaming "save our NHS!!" Not even realizing they are just saving the profits of Accenture shareholders.

A better question is, "How do you spend $1 billion on software and succeed?" The fundamental problem isn't the quality of your developers, it's the fact that you have to coordinate multiple environments composed of mutually-irreconcilable workflow and processes, legacy systems never designed to talk to anything, and real-world problems that aren't amenable to solutions at scale. Big Bang projects are almost impossible to get off the launchpad in a single piece, so can we learn from the ones that actually make it to orbit? Or is there an inverse Tolstoy effect here -- every failed project is alike, but every successful project is successful in its own way?

Rumor has it that Halliburton spent almost $100mil ( or was it $900mil I forget) on trying to do an ERP upgrade so I'm not surprised at all.

It's not just about the money though, it's also about having the mandate to get shit done (and probably hurting a bunch of feelings/fiefdoms) along the way.

Keep in mind, there's not this one magical system that keeps things going that by getting a bunch of really smart guys will right itself.

Often times, it's a combination of COTS and custom software that has grown organically over the years.

So take for example, one of my previous projects, which was to spin up new IT for a PE spinoff. There were over 70 major applications to account for with tons of associated business processes, functionality etc. It was a massive undertaking but yes, it's doable.

How does one spend $1 billion on software and fail?

Epic failure creating sofware is common enough that there's a Wiki page commemorating it. https://en.wikipedia.org/wiki/List_of_failed_and_overbudget_...

My own state managed to piss away around $200 million just on a website to sign people up for health care. It's hard to believe something that insane, but Oracle was involved so perhaps it's plausible.

Well NHS in UK spend £10bn(thats about $14 billion) in failed IT. http://www.theguardian.com/society/2013/sep/18/nhs-records-s...

The industry studies I've seen have shown that, in addition to most enterprise projects failibg, project failure is positively correlated with project size.

My experience in enterprise software suggests that the way many large enterprises -- private and public -- wrap more layers of review, bureaucracy, and big up front design on projects the more money is involved (specifically intended as a risk management technique) is a big contributing factor to this.

So is the idea of a big software project. Almost without exceptional, large enterprise "software" projects really need to be conceived as business process reengineering efforts, where changes to automation are one of means of implementing the reengineering.

Sounds like you haven't read Mythical Man Month. There are a ton of issues that dictate the success or failure of an IT project and money isn't one of them.

You should read the post mortem on the Target Canada debacle to understand the pressures behind a huge software project.

I don't know anything about shipping IT systems but my guess is that it's not just a software in the cloud. You have a lot of hardware involved. Sorting facilities, delivery guys, etc. If that environment is heterogeneous and you need to replace all the hardware or adapt to lots of different hardware, I can easily see how things can go wrong.

The other thing is how can a business person tell the difference between a star programmer and some random mediocre coder? It's not like if it is a profession with reliable minimum standards. It's a bit like psychanalysts, anyone can declare himself a developer.

These are details. The main problem with large projects is that they become complex political games across umpteen divisions. It's futile to think about how you should deliver X functionality if nobody can tell you exactly what this functionality is, who will be forced to use it for what task, and whether these conditions will change in 3 months. The result is an orgy of meetings and committees producing directives that are all over the place, and where interested parties can easily sneak in their own pork and/or agenda.

Any IT project above £100,000 should have a clear line of command, the power to overrule any workflow or operational convention with very little discussion, and freedom to engage in localized feedback cycles without formalized oversight. Pick small and tightly-knit IT/dev teams that can guerrilla their way through the organization with blessed impunity, and just let them loose.

By thinking your problem is a software problem when it's not.

  How does one spend $1 billion on software and fail?

For example : http://calleam.com/WTPF/?p=3474

Found my favorite near the bottom: "Design by committee."

I always find myself wondering this as well. The only way I can think of is they decide to hire thousands of mediocre employees and churn the money away with rewrites and busywork.

Reply on news.ycombinator.com