Hacker News Reader

None of the above changes the discussion. It's a very elaborate way of saying "writing correct software is hard". You know what's also hard to do correctly? Designing airplanes. Building skyscrapers. It's not that their designs are made flawless, it's that (in theory) the sheer volume of man-hours spent pouring over every minute detail of them, incorporating lessons learned from every failure that's ever happened in the field, amount to a very very high success rate.

In software the mentality is that tests, or formal specs, or even thoughtful design, are luxuries. That it's better to just hire someone who can make the computer do things than spend twice as much on someone who can write "good" code and take twice as long to do so (whatever "good" means, because we have no standard). Of course for "the Uber for X" those standards may be fine. But for autonomous vehicles, or credit bureaus, or the nascent private space industry, they are absolutely not.

Perhaps one of the reasons for this is that software doesn't allow humans any way of leveraging our evolved "danger detectors".

It's hard to make a safe building, and it's hard to make safe software. The difference is it's (often) incredibly obvious that a building isn't safe the moment you step into it. In software it's sometimes pretty much impossible to tell the difference between a stone temple and a house of cards.

That's not really true. Much of what makes a modern construct safe is extremely unintuitive; material science being a big example. Some modern buildings are even designed to look precarious on purpose, for aesthetic effect, despite being very sound.

It is incredibly obvious to people with the knowledge and skills. They can look at code and just by glancing at it can have a gut reaction that triggers red flags. Sometimes it can even be an emotional disgust. It's just not common knowledge. We as a community need to do a better job at transferring this knowledge.

Proper strong typing, not just the weak typing that is more common, provides these "danger detectors". It makes it a compile error. Compiler errors make things immediately obvious.

Type systems are very rich and most statically typed languages that are commonly used now can barely count as type systems when compared against what we know from mathematics and computer science. They merely describe the data format (or not in the case of nullable) and not acceptable inputs. They don't allow types for scenarios like requiring a function that exhibits these behaviors but not these other ones.

Pattern matching in combination with strong typing requires code that handles all possible inputs or you get a compiler error.

Dependent typing requires inputs to be within certain tolerances (not just adhering to a data format).

Type systems can also make certain race conditions impossible to create.

> The difference is it's (often) incredibly obvious that a building isn't safe the moment you step into it.

This is just not true. If anything, our "danger detectors" probably give off an equal amount of false positives and false negatives when judging the structural integrity of objects we're unfamiliar with. And the same holds true for code bases, which is exactly why it makes sense to hold software to a similar standard as physical engineering.

> it's that (in theory) the sheer volume of man-hours spent pouring over every minute detail of them, incorporating lessons learned from every failure that's ever happened in the field, amount to a very very high success rate

and this is what company executives need to understand. Given the fleeting nature of software and the speed in which one can write working code (not strictly correct or bug-free, but something that apparently works) people are convinced that software development should be cheap -- or cheaper than say airplane design. Software design is thus seen as cost not investment and those flaws in specifying and writing correct core is then shown in not treated edge cases, sometimes catastrofically

The problem isn't that executives don't understand, it's that they don't care. Capitalism's incentives do not encourage the practice that's best for society. The only industries where care is taken are the ones that have been regulated to be as such. And then there are cases like Boeing, where that regulatory structure decayed and the safety practices decayed in tandem.

This reminds me of one of the first conversations I had with my manager when I started my current job as a software engineer.

I used to work at a large financial company who had an intensely rigorous approach to verifying it's software in both manual and automated ways, as well as a much more careful approach to writing code to begin with.

When I started here (a much smaller startup delivering marketing software for small business clients), I brought up that I was concerned about going to somewhere with a less comprehensive testing pipeline in place, to which I was told:

"It's not particularly important to be able to find bugs before they're released, it's more important to optimize the deployment speed so that we can release the fixes quickly"

Sounded crazy to me at the time, but I later realized it's really all about what your industry is optimizing for, and what would cost you more money.

> it's really all about what your industry is optimizing for, and what would cost you more money

Sure, those were not critical systems, they could be patched with minimal hasse to clients. However you can't really predict catastrophe costs. It could range from minor usability issues to hard to trace calculation errors that could jeopardise the whole production operation. If the company doesn't work work with prototypes it should invest heavily in test coverage, integration test and delivery pipelines

I think there is agreement, or at least slowly developing consensus, that some software should be "engineered" and some should just be "developed". But there is not yet really a structure (that I have seen) for declaring which thing your project is doing and setting up your practices accordingly. One reason this is tricky is that many (most?) projects start in the should-be-developed category and only transition to the should-be-engineered category later after they become mission critical in some way. The answer is often to build an "engineered" version as soon as you can figure out you need it. But then that's even more expensive than doing it that way to start, and it's also a really tough call to make because everybody knows rewrites are a bad idea. It's a tough bind for all but the most obvious cases. And then because those obvious cases are fairly few and far between, the best practices and the experiences of most of the workforce are skewed toward software that merely needs to be "developed".

To make this concrete: a couple times now in my career I've embarked on projects where setting up any level of formal verification at the beginning would have clearly been overkill, but where a few years later I really wished I had it for certain important pieces. And now since formal verification is so uncommon in my experience, I would have to play a lot of catch up to do it, even for a project that is clearly important enough for it even from the beginning. I think this is a pretty widespread experience.

A key difference between physical engineering and software engineering is that the economics of over-engineering to improve safety margins are wildly different. For physical engineering, the costs of substantially decreasing failure risk (e.g. via increasing strength, different materials, more complex design, etc) is approximately linear. For software engineering, that cost is something closer to quadratic or worse.

This is intrinsic to the differing nature of the problems. Verifying the safety of a bridge design is computationally trivial, verifying the safety of non-trivial software is NP-Hard. Safety is so inexpensive to add in physical engineering designs that we invest a lot of effort removing incidental unnecessary safety.

> verifying the safety of non-trivial software is NP-Hard.

While true, this depends very much on the design (or lack thereof) of the software system. Many of our software systems are more complex than necessary because they weren't designed or the designs weren't considered carefully enough. When the software is allowed to grow and accumulate masses of code virtually unchecked, it becomes computationally intractable to continue to verify the system.

Much of the software involved in aircraft, specifically, can be reduced to very simple designs and implementations where the necessary safety analysis is feasible.

The present complexity is often in the accumulation of legacy defects and corrections (fixing the right problem in the wrong place) or trying to fix hardware problems with software. I'm not 100% pro-rewrite, but many avionics software groups (that I've encountered) refuse any rewrites on certification grounds but end up with something unbearably complex as a result (simulating hardware databuses from the 1980s in software on a 1 GHz processor when the system was initially targetting a 16 MHz processor).

Try to solve an aerospace problem with this "simple" approach. No, you don't get to add strength willy-nilly, or just swap materials on a hunch. You're working with safety factors which sometimes are less than 1 (come back for references), and those physical properties of materials are notoriously stubborn to changes. You know, some Richard Feynman once quipped "nature can't be fooled, dammit" - that extends to "water boils at 273.16K full stop (standard conditions), no, you're not going to change that".

Yes, you can often do more complex design. Sometimes though it costs you centuries to choose a good combination of existing physical properties (architecture), and sometimes it's "just" decades (Gravity Probe B). Wanna propose similar approach to software? :)

safety factor less than 1? you're planning that the thing will break early?

Liquid fuel rocket engines sometimes could be planned to work under the plastic deformation stresses, not elastic stresses. Given that their life-cycle is measured in seconds, such an approach - which brings some admirable weight savings - could produce results.

It's a very elaborate way of saying "writing correct software is hard". You know what's also hard to do correctly? Designing airplanes. Building skyscrapers... the sheer volume of man-hours spent pouring over every minute detail of them, incorporating lessons learned from every failure that's ever happened in the field, amount to a very very high success rate.

It's not only hard, but the cost/benefit (or the large penalties of catastrophic failure) works out so that it's worth it to go to that amount of effort. The same amount of effort doesn't go into building a backyard garden shed. I'll grant that we've gotten to the point where the cost/benefit analysis demands that we go to that amount of effort in critical situations.

Writing software is not engineering - engineering is building the same thing over and over again and you slowly learn how to do it the right way. There are incremental improvements along the way - stronger lighter materials, better tools, etc. But the folks who built the Roman aqueducts could get up to speed pretty quickly on the Golden Gate Bridge.

The principles of flight don't change. The fundamentals of building a skyscraper or a bridge don't change. There are hundreds of years of learning behind those things that are still relevant today.

Building airplanes and bridges and skyscrapers is not hard at all. People do it all the time with a high success rate. The hardest part is scraping up the capital and managing the logistics. I'm sure none of that knowledge is required to get any kind of engineering certification.

If you redesigned every bridge and airplane from scratch, from first principles, you would have a tremendous failure rate no matter how many certifications you held. That's what software is - every project is a brand new thing that has never been done before - there isn't 200 years of learning to fall back on.

One of the worst things that could happen to the software industry is for bureaucrats and other people who know nothing about software to really believe it is an engineering discipline and try and regulate it as such. We'll all be forced to do a fantastic amount of busy work to meet arbitrary and archaic "engineering standards", and the software will actually decline in quality. You'll be fined $50,000 if you use tabs rather than spaces, or don't use yoda compares in your C. But as long as you are using spaces and yoda compares you won't be liable for a failure. Stuff basically like that.

Maybe when we have 200 years of experience writing software, rather than 50 or so, we can start to think about formalizing it as an engineering job.

>That's what software is - every project is a brand new thing that has never been done before - there isn't 200 years of learning to fall back on.

Sorry but this is just gatekeeping bullshit. Most apps being built today by companies with the budget to build them are TODO apps, report apps, storefronts, big data mungers, CRMs, etc. They all just have different business requirements and domain specific knowledge. To think that every software project can't be categorized is ridiculous.

The firmware in all the electronic gizmos in your life aren't running "just apps". Much of it is poorly developed shoot from the hip code that is riddled with bugs. Some of it is running in cars and planes that manage to kill people because of hubris.

And some of it is developed under standards like ISO 26262, or IEC 61508. You would be amazed at what goes into your steering, braking, airbag, and stability control systems.

You think that's most of what's being developed today? Life critical firmware?

I honestly don't know what gatekeeping means in this context.

> all just have different business requirements and domain specific knowledge

Yeah - that's what I said. You realize that's where all the complexity is, right?

>I honestly don't know what gatekeeping means in this context.

you're trying to create a barrier to standardization. You don't want to codify successful patterns to keep your job coveted.

>Yeah - that's what I said. You realize that's where all the complexity is, right?

That's not what you said.

> you're trying to create a barrier to standardization. You don't want to codify successful patterns to keep your job coveted.

Ugh. I use as many libraries and services as possible so I can focus on writing code when no standard tools exist. That will keep me busy every day forever. If you think I'm out there sabotaging standardization efforts you're delusional. I wouldn't even know how to do that. Slash tires at a W3C convention or something?

You're being extremely uncharitable.

Look, business processes are different at every single company. Especially between companies doing the exact same thing. Which means that the software ostensibly doing the exact same thing at each of these companies is going to vary dramatically. Because the software at a company aligns to the business processes of that company. This is the true heart of Conway's law: "organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations."

And there is no fixing this. It's baked directly into the axioms. If every company in a domain ran exactly the same way, there would be no such thing as a competitive advantage.

Yeah, and a lot of that software sucks because the companies who pay for it to be developed want the cheapest possible thing that just about works and not some ultra-engineered unkillable concrete bunker of an app that costs them 50x more. That was the entire point of the agile revolution - get rid of waterfall methods inherited from classical physical engineering and switch to constant iteration and shipping, with fixes to not-quite-right work being done whilst workers are in the building. Software engineering became less like regular engineering because that's what the customers wanted, and who's to say they were wrong?

You're ignoring cost. Want to build software like we (were supposed to) build 737s -- to always work? Watch your costs go up by (rough guess) 10-100x.

We've generally made a decisive choice for functionality and cost over bugs.

Which isn't to say that I don't agree that there are areas: self-driving cars, airplanes (required here), human diagnostic software (already required here), etc that should have much tougher standards. But across the board ... I don't think people are willing to pay for that.

> there is no magic formula for ensuring your ideas are always sound. If the idea is wrong you can't even write tests to ensure it's correct.

There are two parts to this. A wrong idea as in building the wrong product, and a wrong idea as in building the product wrong. There aren't any mechanical processes to protect against the first, but building the wrong product is rarely catastrophic (maybe only from a business perspective), and even if it is, it is usually not the software developer's fault. But as to building the product wrong, there are processes and tools that can help -- a lot -- like TLA+, which is slowly gaining momentum and has completely changed the way I think about software. Tools like TLA+ are extremely powerful at tackling those "complex, difficult to predict side-effects when composed into complex systems", but they don't replace careful thinking, they only help you with attacking complexity once you have decided to think hard about the problem. Getting developers to think hard before or during the development process -- but not while typing code -- is a cultural shift.

TLA for the curious.

https://learntla.com/introduction/

I've heard the analogy of a software developer to a mechanic, and I like it much more.

Developer = Mechanic

Computer = Car

Make = Language

Model = Platform

Year = Version

A mechanic is licensed by ASE to work on cars in the USA and there are 50 different certifications [1], depending on the type of car/truck/bus. I don't want to see software development go this route, where developers have to get multiple certifications to prove they can work on a language or platform. It would be draining and force devs into silos, even more than they already are. And as the parent comment mentions, software is an "idea" problem and not a structural/foundational one. Unless your developer is a certified genius, what good is his certification?

[1] https://en.wikipedia.org/wiki/Automotive_Service_Excellence

That happens already. As things stabilize you tend to value predictability. You can have 6-10 smart people do the architecture and design work that drives the work of hundreds of developers.

Programmers want programming to be about ideas, but many/most problems are applying patterns to process.

It is already like that in the enterprise, where certifications are the entry point for certain partnerships.

As engineer I find there should be a baseline.

I also don't expect weekend cooks to keep a Michelin restaurant running.

I don't like car analogies.

A software engineer is like a plumber you link up lots of small components, weld them together and then the data flows through the application (at least that is the idea).

I don't like either of these analogies. Work of SE is much more complicated. I guess a developer who just mashes things together on the front end is kind of like a plumber.

However, if you work with the backend, you need to worry about data access, any concurrency issues, performance problems. Generally tradesman follow relatively simple guidelines, which may change at a slower pace....tech on the other hand changes insanely quickly.

You don't "need" to do any of that on the back-end either to launch a product. Not sure what distinction between FE and BE you think there is here. One messes up an application on the server, the other messes up an application on the client. And there are plenty of avenues for being a clueless plumber on the backend, like using Ruby on Rails + "gem install devise" for authn + "gem install carrierwave" when the boss tells you they want avatars, without having much idea how anything actually works.

> I don't like either of these analogies.

I think it's hard to find a good analogy because software is an incredibly organic and artful thing. It can be highly technical but also playful.

I think it's a matter of finding the right people for the right job, and making sure those people have some skin in the game. If there is no incentive to write quality code, or the dev's don't care to write quality code you probably won't get quality code.

Then again maybe we still haven't come across the correct tools for the job that make it easier to produce quality software.

The fundamental architecture of a car has been relatively the same for many years now. To a lesser extent we see that in computer hardware however in software it so easy to invent new tools that maybe the good tools get lost in the mess of it all.

There is another problem, that software developers in general hate maintenance work. People should be working on keeping existing stuff working and well-oiled. We want exciting work where we get to learn some new tech. Nobody wants the janitorial jobs where you're just keeping things ticking along and running smoothly. Ofcource there are people who work on legacy systems - banking systems, old mainframes, etc. We already have so much software thats out there, that just goes unused, and then people "re-invents" it, and often times makes it worse - with DRM, subscription, micro-transactions, "but now its on the web", etc.

I'd argue that there are an awful lot of us who would like nothing better than to spend our time cleaning up the codebase, refactoring risky code, getting the whole thing under test, etc.

Unfortunately, most organisations only grudgingly fund such work, and if I were to spend a significant portion of my time on such work (without being on a team dedicated to a critical modernisation effort), my performance reviews and compensation would suffer accordingly.

Bingo. Hell, I'd love to do nothing but maintenance work, but I'd have a hard time paying the bills that way.

Agreed. Also, there's a big difference in building a doghouse, tent, home and skyscraper or bridge. Not all of them require the same rigor and planing. Beyond this, a lot of organizations don't allow for pushback from development to cover for clearing technical debt.

None of this accounts for the fact that over half the software industry has less than 5 years of experience and less than 25% has over 10. People with less than 5-10 years of experience can be brilliant, but there are things you learn with experience. First and foremost is the ability to push back and say, "No, it can't reasonably be done in X timeframe." Actually using scrum process to break things down in to chunks that can be swallowed. Not allowing mid-level developers to use "enterprise patterns" everywhere, especially when they increase complexity and fail to provide real value to the project.

Preplanning and the level it takes will depend on the scope of the project and the use case. A heart monitor or guidance system isn't the same as an eCommerce or entertainment website. Engineering discipline isn't required for the latter, but is for the former examples.

Reminds me of Michael A. Jackson's paper [0]. He specifically discusses this very same issue. I think it was posted on HN a while back but can't dig up the link.

0. http://mcs.open.ac.uk/mj665/FoSEZurich2010.pdf

Typing systems and language choices are really small potatoes in this field. There are much more scary problems to solve to make software work right.

When you can side channel attack any CPU with a branch predictor and bits get flipped regularly by cosmic rays at high altitude then you need to rethink what software engineering really means. The big problems are rarely actually about the software or proving some algorithm is correct.

Ah yes, real engineers anticipate the side channel attack that can happen when someone releases a butterfly at precisely the right location and precisely the right time so that it flaps its wings that affects the climate 10 years later that affects the cosmic rays that cause electrons to misbehave in all the CPUs across all the Docker containers on all the hosts in all the data centers and availability zones that causes all the right double entry debits and credits across multiple bank accounts and across multiple banking companies to steal some money.

This is a much bigger problem than type safety.

But luckily that is a hardware problem.

In my country, all engineers are protected titles and basically just says you spent half of your 3 or 5 years studying math. Not even algorithms, just math. Makes it very confusing when teens calls themselves SEs

We could start by not allowing pseudo-engineers out of bootcamps.

Yes, specs and tests don't cover everything, however without them software is much worse.

I guess only an increase in lawsuits and product returns will eventually change the status quo.

Why? Because those with CS degrees don’t make mistakes either? Or how about self taught engineers, do they not count?

A problem is software is so easily created, destroyed and refactored. Combined with business interests pushing for all sorts of things midway through. How often are bridges built halfway when their entire spec changes, oh just convert his half done 2 lane bridge to a 4 lane double decker, should be no problem right? That’s how we treat software.

Engineering is a professional title, with a set of requirements that express a certain level of skill, moral and ethics.

Self taught, yes if they can attain the expected level of the Engineering Order.

> the misapplication of the word "engineer" to software developers.

Is it? you didn't say how it's misapplied.