Hacker News Reader

Comment by diafygi | Hacker News Reader

As we cast about trying to figure out ways to make software more secure or reliable, please remember that in other engineering fields (civil, chemical, mechanical, etc.) prioritizing safety and reliability is a _solved problem_.

(1996) https://www.fastcompany.com/28121/they-write-right-stuff

> It is perfect, as perfect as human beings have achieved. Consider these stats: the last three versions of the program — each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors.

> The process isn’t even rocket science. Its standard practice in almost every engineering discipline except software engineering.

The problem is consequences. We had centuries of people dying in bridge collapses before we got our shit together and started prioritizing safety in civil engineering (i.e. engineers and managers going to prison if they don't).

The same will be true for software. As more people get harmed by thrown together software (e.g. mass panic in Hawaii, state psychological exploitation on social media), we'll start regulating it like other engineering fields.

As a former chemical engineer, I welcome this transition, but I realize it will likely also take centuries of hard lessons.

View on HN

Follow the money. If people were willing to commit to a specification with the same level of precision as a civil engineering blueprint and then stick to that specification then you’d see a dramatic uptick in software quality, along with a dramatic uptick in price and time to develop. But since hardly anybody wants to commit to that kind of precise design or pay the cost we get what we have today instead.

Come to SE Asia and I’ll show you plenty of civil engineering projects with similarly poor planning and execution.

There are software engineering fields where people commit to precise specifications - they typically interface with humans on some biological level (medical devices, etc), and they are quite expensive.

Of course, the same goes for military/infrastructure (self-driving trains) software.

And there are solutions for improving software safety, it starts by improving the process of software development. And for that there's at least one sort of official framework: CMMI.

(Even though DoD stopped requiring CMMI, but it still considers it a big plus, and companies for important stuff all use it.)

Of course, it'll be interesting to see what goes on with the F-35, as a lot of testing is now "skipped". (Sure, that is not software testing, but still, I can't imagine the software dev process is in a better shape after all these delays, cost overruns and probably still ongoing requirement changes.)

I read an article many many years ago explaining why software engineering can't be compared to mechanical engineering.

I can't find it now, but one of its points of comparison was: if a nut isn't tightened enough on a bridge, you can tighten it a bit more -- turn the nut 3 or maybe 6 millimeters more. In software engineering, if you tighten the nut only 3 millimeters instead of 6, one of the bridge's endpoints now ends in a different dimension.

I think there is a valid point to it. I'm always amazed in housing construction how things are done "approximately". A wall should be vertically straight... give or take a few centimeters. That kind of approximation doesn't work in software engineering. It has a whole other type of complexity to it.

If you're building houses, and your walls are a "few centimeters" off from straight, you'll get a real asswhoopin'.

Especially if you're building something with more than one story, and you expect people to actually pay for it.

I’ve never found a perfect 90 degree angle in any house, and I’ve built portions of quite a few. Wood expands and contracts with the seasons.

Which also makes it a really silly choice for actual construction.

Brick and concrete make a lot more sense.

Expanding and contracting is good. Steel also expands and contracts. Bricks and un-reinforced concrete crack and crumble.

Odd how this 1930s brick building I'm in right now hasn't crumbled apart, or even cracked from settling to any noticeable degree.

I'm guessing you've not been alive long enough to see the required maintenance that has been preformed on said building. People tend to miss and ignore things that aren't part of their profession

I have bricklayers and carpenters in my friends and family, and I've been on the association board for the building, for the last 10 years.

So I know quite well what kind of maintenance is needed. We replaced the entire 6000sqm tiled roof last year, it was the original roof, 80 years old.

The maintenance needed on a brick building is fraction of what's needed on a wood building.

Yeah, "a few centimeters" is a lot, but under 1/4 inch is close enough for most residential framing. Windows and doors are always shimmed to fit the rough framing for this reason.

That's why they make protractors for measuring and cutting external baseboard trim. I've seen a few 90 degree corners, but not many.

There are automated methods for determining if a bolt has been fully tightened, and they're generally specified for 100% of the bolted connections.

(The most common one is breakoff studs on the bolts, which actually speeds thing up because the wrench can grab the nut and the stud on one side of the connection and torque away till it pops)

Yeah, I've completely stopped buying this idea that software engineers just need to do what all the other engineers do to improve. Let me know when someone engineers an n-dimensional bridge.

For that matter, let me know when your engineers have built something as complex as the AWS infrastructure, by which I mean, every single service they offer, by information content (something like Kolmogorov complexity). There are some disciplines like power management that certainly have some very large and impressive things that I don't wish to diminish, but the idea that software engineering needs to go learn from all the other engineers is an idea stuck in a very 1970s view of what a "large project" looks like.

I am abundantly confident that if you took the amount of engineering done to build something like the Golden Gate bridge today, then took all those man-hours and saw how much software you could get for the same amount, that you'd be surprised how small the result is. Not that it wouldn't be a good chunk of hours to play with, and it would be much larger than many phone apps or web sites, but if you stack it up to something like a browser or a usable OS kernel I'm pretty sure you'd find that the very visually-imposing bridge is actually orders of magnitude, plural, simpler. I say this not because I don't respect civil engineers, but because I also respect just how quickly big computer projects can chew through the man-centuries. No bridges would get built if each one individually required engineering effort commensurate to a new web browser.

It's past time for programmers to get over their inferiority complex. A sober look at what the programming world does vs the other engineering fields shows that all things considered, we're doing pretty well. Still a huge amount of room for improvement, but we're not doing so badly that we need to go running off to other completely different disciplines to drag in irrelevant, if not actively harmful, practices that are unconnected to the problems we face.

I think I agree that software engineering is more complex than building bridges if you take two comparable project. But I don't agree that software engineering is more complex than building rockets which requires - besides a lot physics know how - also deep understanding of electronics and hardware systems. And if we look at the Ariane 5, the software engineers failed.

An even more demonstrative example, the STS might (still?) be the most complex physical system ever.

The orbiter software group managed to not kill anybody with their software although there were some 17 bugs flown. The same cannot be said of the mechanical and human management systems, which killed one crew and possibly two.

Real software projects often fails for similar reasons, when somebody says "we know it's not designed for that condition, fly it anyway".

Rockets reliability is largely held back by their extreme trade off toward maintenance vs robustness. When you have to check everything and replace a bunch of things between flights, you're bound to miss something.

And that of course impacts the software side too, the rocket is constantly changing, the software has to change too, bugs are guaranteed.

The difference is that while you can’t make a bridge in your bedroom you can make an app.

Should we forbid people from writing code without the proper certification? Should we close down the open internet and replace it with a regulated zone where only compliant software can be run?

I agree that we need a higher standard of engineering in software, but I’m not clear on how to achieve it without draconian measures.

I made a lot of bridges and cranes in my bedroom. They all failed spectacularly but GI Joe didn't seem to mind that the shear strength of a Lego pin was inadequate to support his tank.

The difference is that I don't then promise a municipality that I can build them a bridge that can take their citizens across a river, even in case of a 100 year event. Or an early warning system that can save their citizens from nuclear holocaust. This is the difference in maturity between software "engineering" and real engineering disciplines.

You're free to experiment with your own resources but as soon as you make a promise to the public or your customers you should be required to meet your promises in all circumstances.

This is a fine sentiment, but it's hardly a bright-line rule. We don't criticize Twitter's handling of abuse or fascists because they promised us a platform free of abusive fascists; Twitter didn't promise us shit except 140 character messages. All the problems were emergent.

So many of the great (financial) success stories of our sector are about startups stumbling into an untapped demand, and then running with it for as long as the money lasts. Nobody sets out to build a bridge – they build a 2x4 plank, and then realize that rather a lot of people want to walk on it.

> This is the difference in maturity between software "engineering" and real engineering disciplines.

That's a difference between contractors, people, groups, teams, projects, etc. It has nothing to do with the "disciple of engineering" software or otherwise.

I've seen shitty mechanical engineering projects, but hey, it was cheap, so I don't blame the customer! (We're fighting the shit someone churned out from the software side.)

No one expects to go to a mechanical engineer and ask for a new car, 4 wheels, blue color, 200 horsepower, and a week later "could you just make it 8 wheeled, thanks" and a week later "could you also make it so it'll be fast even when run on a mix of vomit and vodka", and so on, yet it's the norm in software.

And we somehow compare it to bridges, because a lot of software is "critical infrastructure".

Put the two together, and it's no surprise it seems that software engineering is not mature enough.

It simply means some kid (or a team of expensive consultants), who knew nothing about long term risks of the combination of attention fatigue and shitty UX, said yes. Lowest bidder and all that too. Or because this was mission critical it never occurred to the procurers that a UX guy/gal should look at it.

I think you're totally right about this but I also think it still speaks to the maturity of the industry and discipline of software engineering. We still have a lot of lessons to learn about building software, even in the cases where it is bid out like a bridge or built like an airplane. We just don't have as much experience under our belt. It's an oversimplification to say that software engineering is immature but I don't think it's wrong. I also don't think the current state of software engineering is even a bad thing or behind in any way, I just like to think we have a lot more to learn.

Who is we though?

I mean the problem in other fields were solved by a brutal trade off toward a draconian prohibition. You can't practice medicine/law/civil-engineering without a license in a lot of states/countries.

But of course you can give CPR, you can help your relatives sort out their pills. You can talk about laws/statutes/regulations with anyone.

So where would we draw the line? Maybe official/public procurement projects must have a process requirement clause, describing good engineering discipline (clear problem statement, analysis of the problem, analysis of aspects by the relevant domain experts [such as UI/UX, security, network, hardware, etc.], general specification, implementation plan with risks identified and testing methodology outlined, used development methodology [SCRUM, KanBan, whatever])?

This doesn't require setting up big and bureaucratic a trade guild, but if something bad happens we can look at the [pre]filed papers and determine who fucked up. The procurers, the experts, or the dev team.

Hmmm, I think it's too early to tell.

After all, we've witnessed move-fast-and-break-things social networks (Facebook, Twitter, Reddit) that had no desire to influence society in a big way get hijacked by state actors to achieve political goals.

We're also witnessing toasters having web servers pre-installed on them with no thought into what happens if the company that pushes patches (if they do at all) goes bankrupt, thus creating vast oceans of botnets that the rest of us now have to deal with.

Again, I think there's a long road ahead of hard lessons when we don't take our own power in software seriously.

In the UK, you can build a bridge without certification as long as you have someone certified review the plans and the implementation before anyone else drives on it. I suspect this is similar for most civilised countries.

A (perhaps short-term) idea would be to make software vendors liable, and do not permit them to sign away that liability.

I completely agree. But the vendor situation looks very different for software.

Imagine being the sole person responsible for migrating Linux from ip/nftables to ebtables. You don't know how your stuff will be used downstream. So you license it with text in all caps reminding people that your software is provided WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. And people still use it to build oil pipelines that bleed toxic goop when sent a really weird keepalive message.

I hope that the first place we see an uptick is security. The list has grown rather long: consumer/citizen identity breaches, hospital ransomware, digital asset theft, remote vehicle control, etc. Most security people I know are of the mindset that systems will be hacked. It's extremely.. pragmatic. You can't really fault the people responsible when many companies simply require good damage control over actual security in order to be successful.

But that's exactly the point. If the consequences were increased, not so many C[I]SOs would be okay with engineers using hot new software every few quarters. And those old boxes running Ruby 1.87 would sure as hell be patched or isolated to oblivion. Companies or projects with good security would flourish. Maybe some would be pressured to operate more like the archetype they're defending against (more red team, more organizational commitment to physical and operational security).

I worked at a security company that had to get rid of a slack bot screen lock game because it hurt some people's feels. So yeah I think some of the priorities in this industry are messed up.

Those people who build oil pipelines should be able to sue whoever sold them the Linux distribution (probably Red Hat or IBM) which was vulnerable to that weird keepalive message.

In that case, a judge (and perhaps a jury) could hear how Red Hat did everything they possibly could to protect from the vulnerability as evidenced by their ISO QA processes and the fact that everyone else was vulnerable to the same "bug" … or from the other side how Microsoft and Apple weren't at-risk, so Red Hat should've caught it.

C[I]SOs would want to be patched, because ISO recommends they would be patched.

> You can't really fault the people responsible when many companies simply require good damage control over actual security in order to be successful.

Which is why I propose legislation, so "good damage control" wouldn't be enough.

You better believe that oil company would want some evidence of testing and proper specifications, and to have them reviewed by a couple independent parties if the government could take them for a percent of gross revenue for the security vulnerabilities alone.

I disagree, unless RedHat certified that the software they're selling is of safety-critical quality. If the oil pipeline company used, let's say, nuts and bolts that didn't meet safety requirements without checking that the parts met the requirements or being assured by the vendor that they did, I would say they're the ones liable and not the vendor.

Maybe it would make more sense for people to be liable for data and not code. There's no need for my cute photo effects mobile app to undergo some kind of exhaustive code review but if I want to store credit card data or social security numbers maybe I should have to pass some kind of review.

> in other engineering fields (civil, chemical, mechanical, etc.) prioritizing safety and reliability is a _solved problem_.

One big difference between those and programming is that there (outside of military, physical security, and few other exceptions) there are no active adversaries for your project. You design a X that will serve a defined purpose. You design for some parameters (known wind speeds or local natural hazards).

You don't normally design with someone actively trying to destroy your building in mind. You don't design thinking how to avoid the current issues and to isolate the impact of potential future attackers. Your product doesn't have a "well funded attackers will spend undefined amount of time trying to make your project fail" requirement.

> You don't normally design with someone actively trying to destroy your building in mind.

You do, actually. When designing a building, one considers many "attacks" it can be subject to, depending on the geography and demography of the site: what are the precipitation patterns like, what is the situation WRT sysmical activity, how is the soil, what is beneath, what is the crime rate in the area, what kind of animals live there, where does the wind tend to come from, how much sunlight is received, etc. etc., about all of these questions are in one way or another is relevant to security of a building, and you have to consider each and every one of them, and design accordingly. You don't design against destruction because that's hardly possible, there isn't much you can do to mitigate someone bombing a building or attacking with heavy machines. Think of that as analogous to using the ME in an Intel CPU.

'When designing a building, one considers many "attacks" it can be subject to'

Your use of scare quotes is appropriate, because there is a qualitative difference between intelligently-driven and unintelligent attacks. Not to mention the intelligently-driven attacks when the attackers are manifestly more intelligent and skilled than the defenders! If computer security experts didn't have to worry about intelligent attackers, computer security would be very nearly a solved problem.

I'm no security expert, but this article, and the FastCo article about NASA programmers suggest, at least to me, that computer security is a problem that emerges from incompetence in using tools and designing programs. We're yet to make the differentiation between hacking away amateurishly and building a software solution that will be commercially offered. A showerthough I just though is that only companies registered as software companies should be able to offer software commercially, with relevant regulations implemented on them including mandatory third-party auditing. This is not dissimilar to how any other sector of business works. I don't know US or EU, but in my country, for example if you want to sell foodstuffs, you need a certain certificate; if you want to produce foodstuffs, you need another certificate particularly for that purpose. So if you wanted to open a patissery, you would need a couple licences at least, you'd be subject to some control from the council's related unit, and there are institutions to handle any health problems or bad practices about this sort of commercial entities, however small or big they might be. I can't understand for the life of me why __at least__ the same level of standards are not in place for software that handles my money or my health data or my personal data. It hinders innovation? Well we saw what innovation can do both during the car boom and with Meltdown and Spectre, when nobody's actively, rigidly and methodically thinking about security and integrity.

WRT the house analogy, it's easy to extend that to "intelligent" attackers: intruders of any kind, e.g. robbers, animals, etc. Many install security cams, in my city (Istanbul) many condos have railings that protect the windows of the lower flats, we have locks on doors, alarms, barbed wires, safes, body guards, guard dogs etc., all to stop the intelligent attackers to actually using their intelligence. What's analogous in programming is using the best practices available, and the use thereof must be imposed on any critical systems (e.g. banks, medical institutions, communications tools [e.g. social media] etc.) by the governing bodies.

"I'm no security expert, but this article, and the FastCo article about NASA programmers suggest, at least to me, that computer security is a problem that emerges from incompetence in using tools and designing programs."

While true that accounts for a large amount of the problem, probably the clear majority, computer security would remain a problem even if developers were uniformly highly competent. Competent use of existing crypto systems, which are broken three or four years later, would still be a problem. Meltdown and spectre would still be a problem. Building a safe execution sandbox is legitimately difficult.

But it would be a qualitatively different world than the one we live in.

Certification solutions to the software problem generally face the problem that it is very difficult to imagine any scenario other than one in which people grotesquely incompetent to write the certification rules are the ones writing them. We do not, for instance, want our certification authority to sit there and mandate waterfall design processes, which I would consider at least a 25%-probable outcome, and that's awfully large for something as catastrophic as that would be.

"WRT the house analogy, it's easy to extend that to "intelligent" attackers: intruders of any kind, e.g. robbers, animals, etc."

No, houses are never under such intelligent attack. Even when attacked by humans, they are not attacked by ninja stealth thieves who go in, photocopy your SS card, and get out again without leaving a trace or something that sounds absurd to even use as an example. There's no physical equivalent to breaking into millions of houses at a time and making off with such data. They're attacked by people who smash through the physical security. Anybody can do it. "Anybody" is who does it... above-average IQ people are not generally breaking into houses. (Above-average IQ criminals find much more lucrative and safe criminal hobbies.) Not just anybody can put a tap on a fiber optic line, feed it to a high speed data center, and process it in real time to extract out terrorism threat info, or even just exploit an XSS vulnerability on a website.

I specifically said that physical security is an exception and "You design for some parameters (known wind speeds or local natural hazards)." These are all initial assumptions and known ahead of time.

> As we cast about trying to figure out ways to make software more secure or reliable, please remember that in other engineering fields (civil, chemical, mechanical, etc.) prioritizing safety and reliability is a _solved problem_.

The article makes an unfair comparison; if the solution to the problem is like the one below (extract from the article), the change, reasonably, will never happen in commercial programs (the ones "with 5000 errors").

> Take the upgrade of the software to permit the shuttle to navigate with Global Positioning Satellites, a change that involves just 1.5% of the program, or 6,366 lines of code. The specs for that one change run 2,500 pages, a volume thicker than a phone book. The specs for the current program fill 30 volumes and run 40,000 pages.

You over-estimate the degree to which the other engineering fields you mentioned have solved safety and reliability. Just to name a few:

- Tacoma Narrows bridge: https://en.wikipedia.org/wiki/Tacoma_Narrows_Bridge

- Thalidomide: https://en.wikipedia.org/wiki/Thalidomide

- Hyatt Regency walkway collapse: https://en.wikipedia.org/wiki/Hyatt_Regency_walkway_collapse

- Challenger disaster: https://en.wikipedia.org/wiki/Space_Shuttle_Columbia_disaste...

- Columbia disaster: https://en.wikipedia.org/wiki/Space_Shuttle_Columbia_disaste...

- Vioxx: https://en.wikipedia.org/wiki/Rofecoxib

...and plenty more. The only thing that separates software engineering with the other engineering disciplines is that there is a structure (usually in the form of code/spec enforcement) for internalizing and learning from disasters when they happen.

The Tacoma Narrows bridge collapse happened 80 years ago, the Hyatt Regency was nearly 40. When did you last read about a software bug?

Phones that suddenly combust came out in 2016. The Big Dig ceiling collapse was 2006. Rana Plaza collapsed in 2013.

Serious engineering failures are still happening. Sure, the most famous ones may be decades old, but those aren't the only ones.

Professional engineering has plenty of recent disasters to point to, whether failures in design, construction, operation, or security:

The Waco fertilizer plant explosion (2013), cause determined to be arson (2016)

San Francisco Bay Bridge seismic bolt failures (2013), caused during construction: https://www.nace.org/CORROSION-FAILURES-San-Francisco-Bay-Br...

The Fukushima Daiichi nuclear disaster (2011), design/regulatory failure

And so forth: https://en.wikipedia.org/wiki/Category:Industrial_disasters_...

And so on: https://en.wikipedia.org/wiki/Category:Engineering_failures

From the article (1996):

And it’s the dominant image of the software development world: Gen-Xers sporting T-shirts and distracted looks, squeezing too much heroic code writing into too little time; rollerblades and mountain bikes tucked in corners; pizza boxes and Starbucks cups discarded in conference rooms; dueling tunes from Smashing Pumpkins, Alanis Morrisette and the Fugees. Its the world made famous, romantic, even inevitable by stories out of Sun Microsystems, Microsoft, and Netscape.

It's incredible how much software development has changed over the last 21 years. Change Gen-Xers to Millennials, update the playlist, and replace Sun Microsystems, Microsoft, and Netscape with Apple, Facebook and Google.

I can't tell if you're being sarcastic or not in your second paragraph. :-)

That is my point of view as well, software needs to be liable just like in other fields of engineering, done by people that actually understand its consequences.

I don't do a car inspection at a guy that just happens to know some stuff about mechanics.

I think the difference is that in engineering you are legally obliged to follow this process and in software you are not.

The leads to a race to the bottom for which both the software vendors and consumers are responsible. Everybody wants the latest stuff as soon as possible, and wants to play less. If you release software only when everything's "right", the market will have moved on.

> If you release software only when everything's "right", the market will have moved on.

... unless we the people demand legislation that protects us from software "vendors" selling us their bugs.

I think it's interesting that people can see a car that doesn't work, and a house that's falling apart, but they can't really see software. Can you imagine someone saying "my house is leaking" and the landlord saying, with all seriousness: "Have you tried turning it off and on?"

Of course, we have legislature for those things, so why not software?

I am sure other fields have a better development story these days, but isn't it a matter of perceived risk and relative effort?

I.e. we rarely hear about plane crashes caused by software, or power plants accidentally blowing up due to an integer underflow.

Civil engineers make solid bridges, but most software is not the golden gate, it's a shed thrown together by some guy who's not an engineer nor a professional mason.

How many people die because of software? How many die because of cars?

We could make cars that don't kill people.

We could make software that doesn't break.

Reply on news.ycombinator.com