Why can't the IT industry deliver large, faultless projects quickly?

Two years later, they get breached, 10 million unhashed passwords leaked, and yet they have such a stranglehold on the market that the breach barely affects their bottom line.

Compare that with say, the aerospace engineering industry, where putting out a faulty airplane that crashes, brings with it lawsuits, damages from victims, huge hikes in insurance premiums, and higher demand from consumers for your competitors' models.

Examples:

Bridges that have cracks. Roads that become too slippery when it rains. Railways that require immediate reconstruction after the first month of operation. Delays, environmental disasters, multi-million projects that become multi-billion projects. Projects that are ten years late...

The Airbus A380 mentioned in the question probably has thousands of defects but it's still a very successful project. On the software side I could mention Windows 10, most triple-A games and Tesla's autopilot. Of course these products have defects. But they're still very successful projects. Give them some credit.

In other engineering disciplines, failing is often life-threatening and the cost of fixing the problem can skyrocket pretty fast.

So, in IT, faultlessness does not weigh very much against time-to-market or development costs.

In projects were IT faults could be lifethreatening, you generally see much higher quality standards

In my experience some or all of the following will happen on big software projects...

The sales team will bid unrealistically to get the contract, either by underestimating development time or by bidding too low. They will also ignore the advice of experienced devs an PMs when planning their bid.

On winning the bid there will be a ruthless focus on doing things as cheaply as possible. A PM with no experience of this type of project will be put in charge. The rest of the team will largely be chosen based on availability rather than skills or experience. The positions of software architect or technical lead will be seen as optional.

Whatever PM methodology is chosen (probably Scrum), no one will really understand the finer details of how it works. Work will be divvied up poorly, e.g. tasks that need two people working on it together will be given to one person and vice versa.

There will be no thought given to training. Devs will arrive in to find out they are working on some hyper complex financial system, but they will never be given training on what that system is for or how it works.

Inevitably the specs as defined in the bid document will be discovered to be incomplete or faulty. No one will want to actually change the specs, but somehow the devs will be expected to just deal with it.

If the contract is time and materials based as opposed to fixed price then the contractor will massively over-resource the project meaning you end up with a too many cooks situation.

The client will get wind that there are problems and raise concerns. The PMs and hgh-ups in the contracting firm will claim everything is Okay and demand devs work overtime to get things right. They may also utilize smoke and mirrors to fool the client.

As stress builds among the PMs they will start to micro-manage, this will alienate senior devs who know they don't need to take that and they may leave for pastures new. Junior devs will carry on despite still not having the required skills or experience. New devs will be hired, only to leave soon after when they realise that the place is in a mess.

Eventually the project will be delivered late and full of bugs, yet somehow the contractor firm will claim everything went great and organise a piss-up to celebrate.

1) Not empowering developers by blocking access to critical resources, basically walling them off in a tightly-controlled box and still expecting them to do their jobs because "they should be able to figure it out". Ridiculous shit like no admin rights, strict enforcement of specific tools,etc. This is usually due to an IT or security department with a inflexi,no fucks given, block everything attitude.

2) Not hiring a strong architect who has the vision for project and the power to make major decisions. So many organizations don't think this is necessary and often have someone in management play this role.

3) Having a weak management team that won't stand up to the customer to tell them what can and can't be done, and not letting them dictate how to do technical things.

4) Skipping or not enforcing basic well-established engineering practices such as design, testing, peer review, etc.

5) Using ill-advised "solutions" platforms that are not intended for engineers but for business analysts with some tech background. These platforms are sold to government and enterprise by used-car-salesmen-like sleazy marketing departments that straight up lie about capabilities or omit huge faults or inabilies. Stupid and/or tech-ignorant management buys this shit and it's crappy, expensive licenses for years to come. They can't get rid of it because of sunken costs, etc.

6) Fear of anything open source

7) Inability to find good, reliable talent -- anything from engineers to testers to tech writers (and who actually have good English skills). Many, many people I work with are great engineers but have such poor English skills that I literally can't understand a thing they're saying, even in an email.

8) Management or stakeholders straight up not giving a shit and letting the project die due to personal problems, infighting, interpersonal issues, corruption, etc. This I would say is the most important issue.

Organisations often fail to realise the strategic importance of big IT projects, and fail to treat them accordingly. "The geeks" are tasked with figuring it out.

Whenever a new subway line is built in a city, years and years are spent moving existing underground infrastructure like sewers, water, gas and electricity. All sorts of weird complexity tends to bubble up around this time. A comparable IT project will typically have requirement 264/B-4: The project will be delivered without changing existing infrastructure.

This again probably has something to do with the sequestration of IT as a staff function. This is absolutely the correct place for the office that builds and supports workstations, runs the Exchange server and manages the Active Directory, but it is not where you anchor a large, strategically important project (IT is definitely an important stakeholder, but they don't own it). Emirates doesn't have a staff office tasked with buying, taking delivery and integration the A380 into the fleet, that is a project that spans the organisation and is run out of the CEOs office.

Tangentially to this, there are the complications around the fact that many large IT projects replace existing systems, sometimes decades old. I'm not sure there really is a good physical analogy to the ways old IT systems creep into and get entrenched in adjacent use cases, often without the system owners really realising. It's like Emirates realising two weeks before the inaugural flight of the A380 that there is something about the ailerons of the new plane that means half of the check-in staff can't get to work.

Then the questions are what exactly constitutes a project and what are failures? Taking the example of the quoted A380 example. Deliveries have been delayed: https://www.bloomberg.com/news/articles/2016-12-27/airbus-to...

And the engines the mainstay of a plane have found to be have problems: https://www.bloomberg.com/news/articles/2016-11-18/emirates-...

With a mechanical part it often is obvious to see and easy to understand what this part does. A lot of engineering goes into finding the right materials, the best shape and processing tools. But the finished part is easy to understand.

With software engineering we have abstractions like "sort algorithm" that are easy to understand but many things have so many cross-connections to other parts that in order to understand what is actually going on you have to get really deep into this stuff. Changing a detail in one implementation may lead to unforseen consequences on other, seemingly unrelated, parts. You may rely on undocumented features without even knowing that.

How many cables do you see when you cut a boeing in half? How many of them are essential to operating the plane? How many dangling pointers do you get if you cut your main memory in half? How many of them are of system components?

Whatever technique we have to cut down complexity, hide implementation details behind clear interfaces, separating stuff into easily-to-combine modules makes managing projects easier. I think that the main success factor of python lies in this area.

Then imagine a project that does not have all those modules readily available. That does not have engineers who know it all (and really do, not just pretend) in charge. They rough out a plan and start working on many ends that - when they meet - do not fit together. Add in organizational, personal, psychological problems and the structure wraps in on itself, turns on itself in infighting about how-to-do-stuff, career postions, blame, praise, just surviving or just passively standing by fulfilling orders on paper waiting until everything collapses.

1) via tenders, where it's a race to the price bottom, and the vendors charge massive sums for change requests or simply to fix what should have worked in the first place, but the government must pay additional money in order to not have wasted the entire sum already spent

2) tenders with specs that are customized so only a select few (or one!) vendor hits the criteria (e.g. must have 20+y of government work history, x% of workers must have top-secret clearances) or lock-in; basically "no one ever got fired for buying IBM/Accidenture". The big vendors of course know this and charge govt through the nose.

Another reason especially in government is a lack of client-side controlling, i.e. there is no qualified staff to review code, architectural decisions or cost/time plannings early (if at all). This also hits home for construction projects.

As a purely digital artifact, it is highly malleable. Yet its deeply abstract nature, and the complexity enabled by those abstractions, make software very difficult to inspect and understand, compared with the vast majority of things humans make.

If roads were tested like software, you could say "yeah, it supports all the common/pre-defined use cases, but we just found out it's susceptible to a jackhammer. Better fix that." And then your road would never be finished just as software is never finished.

Now throw in a good measure of politics with a dash of bureaucracy and you have your answer.

http://www.projectcartoon.com/pdf.php?CartoonID=2&PaperSize=...

This can be broken into three levels of hierarchy.

The executive level is the level VP and above - they write checks. If you don't spend the company's money - you are not at this level. They spend money on programs. They don't know or need to know (or care about) specifics. They are only concerned with the investment and not even individual investments but the organization as a whole.

The bottom level is the IT worker. They are the programmers, managers, etc.. that carry out the specific tasks, do the work.

In between is the (often called Sr. Managers, Acct execs, or VPs in consulting). Not straight sales guys (which is a specific skill, and who are specifically there to close, that's a different breed).

These are the schmoozers. Project people (level 3) have to report to them and they typically hold pretty decent amount of power (reporting to the CEO of a smaller IT consulting or svcs firm, or a COO or SVP in a larger firm). They get credit, sometimes deserved, for bringing in business, so they have power - AND they are often completely incompetent in knowing what to look for and managing a project to success.

They have the power and they don't know what to do, or won't do it. The managers below them, experienced team leads, etc.. know what to do but don't have the power or resources under their control.

It fails at this level, and the level 3 folks (proj mangers, leads, dev, etc..) are not directed appropriately. They may be the greatest team leads and developers etc.. in the world, and the project still fails.

That is we often have a proof of concept ready or the skeleton and then declare it 80% done. Then we extrapolate and say, "well I'll a bit more just to finish the last 20%". Except that the last 20% is composed of both boring and unpredictable things. Boring like having to write tests, documentation, find out that it doesn't jive with the deployment hardware, plugging logging, authentication into it, managing upgrade from old versions of software, having meetings with marketing. Unpredictable because in that perceived 20% requirements change. Even more insidious is the market as a whole change. Ever seen it happen - while finishing a major project and about ready to release it, everyone moved on and nobody wants that kind of a thing anymore? I've seen it.

Being late on an aircraft or building probably comes with financial penalties.

The question uses Airbus A380 as an example of a complex project that was delivered in just five years. But according to Wikipedia the project started around 1990 which adds 10 years to the timescale, and the project actually had multiple delays https://en.wikipedia.org/wiki/Airbus_A380#Production_and_del... and multiple issues were found (and fixed) after delivery. This is not at all surprising given the scope of the project, but it is strange to present this as delivering a faultless project quickly.

"Airbus cited as underlying causes the complexity of the cabin wiring (98,000 wires and 40,000 connectors), its concurrent design and production, the high degree of customisation for each airline, and failures of configuration management and change control" - does this sound familiar?

Of course, when the model have been designed and tested, it is pretty predictable and low-risk to build the next one to the same specs, just like it is pretty predictable to build a standard building according to well-tested plans. But this is comparable to installing an existing tested software package on an new computer, which is also pretty predictable.

Conversely, in a software project neither of those things may be true. Imagine the "reset password" feature. Sure, it reset the password for your test account. What about a student account created before 2010 (when there was that big accounts change) with restricted ads, from China, marked for suspicion of fraud and the account details service is responding slower than usual... plus, the feature was written by an intern in his second year of a CS degree.

Any aviation or civil engineering project is in a highly regulated environment, and lives can be spectacularly lost if the product fails.

Lawsuits and criminal charges would likely be large and numerous if such a project failed, and was built with entirely new processes and technologies that were seemingly chosen on the whim of a project leader and their people, combined with "something we found on github." (I'm guessing there's no such thing as github for civil or aerospace engineering.)

If your environment constrains you in some dimension, then you're not going to spend much time and resources discovering new ways to stretch that dimension.

"Cherish your constraints," if you will.

How many software projects have that sort of scope?

Unless you're working at Google scale, what's the budget for your website project? $100k? $200k? Departmental budget of $3million a year which is 90% salaries and 10% hardware?

Generally speaking, those businesses handling airplanes, submarines, skyscrapers etc are always working at that scale and everyone is prepared for that.

Companies that bid, also have twisted priorities, the people who do the bidding, aren't the people that have anything to do with the project, they more onto the next bid.

But the difference is likely that:

1. Time frames are quicker in IT fields, especially web development fields. So corners are cut and things are rushed to get them out the door. Same deal in games, where publishers assume they can release now and patch later.

2. There are stricter regulations in other fields. Not surprising cause they need them, the consequences are higher if things go wrong. Meanwhile IT projects (especially on a smaller scale/outside the finance industry) can be summed up as 'if it works, it's good'. Same with game development.

3. Human attacker risks are lower in many fields. This comes down to two things:

A: There's less of an incentive to 'test' non IT projects. For IT stuff like sites, breaking them can mean money from bounties (at the good end) and data to sell to criminals (on the negative end). For games, it can mean distributing pirated copies via torrent sites or gaining access to the console hardware. For an aeroplane or building? Why would anyone need to find faults? At least outside of internal testing or certification services...

B: It's much easier for them to find faults. They can attack an IT project without leaving the house, and they obviously have physical/file access to any games they own. No one has easy access to test buildings or large vehicles.

4. The risks provide a greater incentive to 'do things right'. If the worst that can happen is a site crashes/gets hacked, or even that the user finds a few random small bugs, there's less of an incentive than if the risk is loss of life or a huge financial loss.

5. A lack of standardisation or 'best practices'. People know what's good in vehicle or building design, and most are very much like others of the same type. Websites are mostly unique from each other, and games are often similar. There's a lot more uncharted territory there.

6. Careful planning is more common in some fields than others. A plane or building is probably not going to be revised very much, especially if there's no pressing reason to revise it mid development. A website or app may well be (especially if the client keeps changing them mind), and a game often will be as well (if something either isn't turning out to be fun, or the management decide the latest hot idea has been incorporated now). So the time available for testing and proper development can be limited due to constantly changing demands.

And yes, I feel the 'managing complexity' aspect brought up by static_noise is relevant here too. It's easier to figure out what each part of a mechanical system does than it is to do the same with a computer program or script. That's why large sandbox type games (like say, Zelda Breath of the Wild) can get rather buggy if you try unusual things. Because no Q/A team on the planet can find every odd thing a player could try and get the developers to implement a logical response to it. No one's going to find stuff like that which Stryder7X shows in his videos without being either going well out of their way to try insane things or being shown them:

https://www.youtube.com/watch?v=XFWjGXmnpno

Either way, there are lots of reasons really.

Another issue is simply encapsulation. You can encapsulate physical problems to reduce their complexity. A spring is a spring. We "could" use finite-element-analysis to make a spring that is also a damper, and a lever at the same time. But it's way more cost-effective and easier to reasonable when we use discrete components. A spring is a spring. A damper is a damper. You want both, you put them together.

We "could" create capacitors that are also inductors and resistors. But it's way easier to reason about them separately.

Like I said, a spring is a spring. It will never set off an atomic explosion. It will never turn into lead. It will never warp into a living person. You can reason about that spring and as long as you live within "safe" areas it will always act the way you expect it. You can easily predict the failure rate of a given spring for given environment of heat, forces, etc.

Meanwhile, you have an app. You think it's encapsulated. But it's not. Your app is a network app. That means the router could die or be misconfigured. The ISP could go down. You could forget to pay the ISP bill. The internet backbone could go down. And, not only that, it's not just "down" it could simply be "slower" or have a higher packet loss than normal, or end up sending duplicate packets.

We spend so much time trying KNOWING that things can fail anywhere along the line, we "fix" it by adding redundant, fault-tolerant, load balancing systems. Except each of those systems increase the complexity and can fail in even more ways!

And I haven't even discussed what the app does yet! And just being connected to the internet or another network machine and it's already exploded in ways that it could fail. Even the WIRES in the wall could be corroded, unplugged, or eaten by a rat. And, if the power goes out anywhere along the chain you could have problems. If the power "kind of" goes out and you don't have proper battery backup, it could introduce errors. A Windows Update can (and DOES) break products all the time. A Java Update can break things. A driver update can break things. A hard drive failing can subtly break your software. Any third-party software you use could have a hidden bug. A router could have ONE and only ONE port, or one cable with a slight kink in it, could have a higher error rate.

There's a famous story of back-in-the-day when mainframe actually failed at the same time every day. It failed when people walked over a specific TILE in the floor where the wires ran under it.

Meanwhile, that spring, sitting on a table in some warehouse will remain a spring till it rusts away a couple hundred years later. The spring constant won't change. It won't unravel. It won't leak your boss's e-mails. It'll still be a spring.

So while yes, there ARE very complex mechanical constructs. However, they're actually pretty rare because in the physical world "simple" is the most reliable and often the strongest solution. Meanwhile, in business, you're dealing with tons of logic and even though you "think" it's encapsulated, it's actually not.

Summary / TL;DR: Most mechanical machines aren't dealing with the dimensional explosion of possibilities. They do have failure points just not as many.

(Also, one great thing about many machines is because they're physical, you can often diagnose many of them before they fail. They vibrate. They create sound. They output less than they're supposed to. A router doesn't warn you before it decides to stop forwarding a port.)