Comment by ssivark | Hacker News Reader

Comment by ssivark | Hacker News Reader

> How did software get so reliable without proof?

Simple, software started shipping with more bugs in backlogs :-P

— More seriously, software today is endemically crappy, and often poorly designed. I dunno why Hoare thought it was any better. The only saving grace is that (modern) software largely stays away from serious stuff. Eg: The airline industry is reluctant to upgrade software from decades ago, but happy to incrementally upgrade other pieces of the system on shorter cycles. Then of course, we have systems like the Boeing 737Max MCAS where the software did what it was supposed to (taken literally) but the software system was poorly designed.

EDIT: Just remembered this fantastic talk by Joe Armstrong https://youtu.be/lKXe3HUG2l4

The smooth running of every abstract system depends crucially on the (human) operator handling the point of contact with reality, in practice, often bending reality to make it tractable for the system. Any bureaucracy would grind to a halt if it wasn’t intelligent humans carrying out the processes! Software is no different.

Just like it’s hard to take a technology from “zero to one”, it is hard to take the amount of necessary human oversight from one to zero. For this reason, I would much rather think of software as amplifying the capabilities of that human, rather than automating away other humans. In practice, these systems will end up needing a highly skilled and trustworthy human operator to shepherd them — might as well design the system to make it maximally easy for those operators to understand/debug/tweak.

> software today is endemically crappy, and often poorly designed

So is every other engineering product that hasn't gone through years (decades) of evolutionary refinement.

Not really. When programming was done by real engineers (rather than "coders", "developers", or horribly misnamed "software engineers") - with proper engineering discipline which involved deliberate design and documentation rather than "agile" hacking, it was more reliable. By far.

My personal recent experience of actually doing software the old-fashioned way involved writing correctness critical high-performance code for a major data warehouse vendor, which mostly stayed with zero known bugs in production - greatly contributing to the vendor's reputation as reliable and dependable place to keep your data in.

And how do I know what the old-fashioned way to write code is? Well, I've been writing code professionally for nearly 40 years.

The more I learn about modern software practices, the more I come to think that software is worse today because programmers are poorly disciplined and badly trained. Of course, that's likely because the barrier to entry is so much lower today than it used to be.

The barrier of entry to software has been zero ever since the 8080 was introduced.

Back then there were a lot less stackoverflow copy & paste mistakes.

There were also far fewer users to really discover the nasty edge case bugs.

> There were also far fewer users to really discover the nasty edge case bugs.

If you consider why a company makes something reliable or not, it's a relatively simple formula:

    = Number of Users x (Benefit of Getting it Right x Probability of Getting it Right) - (Cost of Getting it Wrong x (1 - Probability of Getting it Right)))

As the number of users in any system increases, the cost overall cost of getting it wrong also increases. You can then devote more fixed cost resources to improving probability of getting it right.

> it was more reliable. By far

[talking about non-software engineering] There's a huge difference between reading books about proper engineering and how it is actually practiced. Much of the sloppiness is covered by simply over-engineering it. Designs are constantly improved over time based on service experience. Heck, lots of times the first machine off the line cannot even be assembled because the design dimensions on the drawings are wrong.

The idea that non-software engineering is done by careful professionals following best practice and not making lots of mistakes is painfully wrong.

As an ex-machinist, I can confirm that bad drawings are a thing.

But then, non-software engineering is a wide field. There are products that you can afford to iterate on, and then there are very expensive (and potentially very dangerous) projects where you generally can't afford many slip ups.

If your engineers make lots of mistakes (which aren't caught in time) in a project that costs millions and can't be replaced by another unit off an assembly line, that's kind of a big deal. Thankfully, we don't hear about bridges, skyscrapers, heavy industrial lifts or nuclear power plants failing all that often.

The things you mention are over-engineered by wide margins to cover for mistakes.

The first nuke plants are pretty dangerous by modern standards, and we know how to fix them, but because the first ones are dangerous we are not allowed to build fixed ones.

The Fukushima plant, for example, had numerous engineering faults leading to the accident that would be easily and inexpensively corrected if we were iterating.

Airplanes are a good example of how good things can get if you're allowed to iterate and fix the mistakes.

Old fashioned way?

You mean all that softwarte written in '90s with no security in mind?

I bet you're talking about outliers. Nowadays average developer practices are levels above that what was decades ago while being supported by great tooling.

I feel like there are two separate things going on here.

You can have an extremely reliable piece of software running say, an industrial lathe or a stamper or book printer or whatever - software which can run 24/7 for years if not decades, software which will never leak memory, enter some unknown state or put anyone in harms way - and yet have zero "security", because if you plug in a usb keyboard you can just change whatever and break it entirely. Software which has no user authentication of any kind, because if you are on the factory floor that means you already have access anyway because the authorization step happens elsewhere(at employee gates etc).

It's like people making fun out of old ATMs still running Windows XP, because it's "not secure". If the machine isn't connected to the internet, reliability is far more important - who cares windows XP is not "secure" if the ATM can run constantly for years and reliably dispense money as instructed and there isn't a remote way to exploit it.

I feel like that first kind of software(the reliable kind) is far rarer today - people just throw together a few python packages and rely on them being stable, without any kind of deeper understanding of how the system actually works, and they call themselves software engineers. The "security" part usually also comes as a side effect of using libraries or tools which are just built with "security" in mind, but without deeper understanding what having truly secure software entails.

It really depends on the scenario. When I wrote software for load balancing phone calls, it was minimal, had a well defined state machine, passed all the static testing I could throw at it, etc. At the same time, I wrote some crappy web service code which could fail and get retried later, because nobody would see that. If the worst thing that can happen is that one in a million visitors will get a white page, it doesn't economically make sense to do better. Even if you know how and have the tools.

I would find it very hard to bring myself to do that. I won't knowingly write incorrect code even if the chance of failure is very small. Luckily I don't have a boss breathing down my neck telling me not to waste time.

I don't think anyone knowingly writes incorrect code. But you can spend between 0 and infinite time thinking about whether the code is correct. At infinite you never release, so you don't need a boss to have some reasonable time limit. If this is your non-hobby work, you need to decide when to stop looking and accept the potential issues.

I doubt software engineering is engineering discipline. At least not the way we are doing it now. We are not engineers, we are craftsmen. It is a miracle that the software that drives the world more or less works. So far there hasn't been any major catastrophe due to software but it will happen. Software sucks.

Over time, I think I am coming to agree with you.

I went to an engineering school. People who designed bridges had a wide, but constrained parameter space, and well-accepted design patterns.

I started out in semiconductor (MCU) systems and sometimes circuit design, and we had a broader (but not yuuuge!) parameter space, but it was growing as transistors got cheaper. Less well-accepted design patterns, because what you do with 50K transistors and 500K transistors and 5M transistors and 50M transistors - to use effectively - you need different patterns - and that changed so fast!

I did software-ish things with my h/w teams, and they would mock me because "software is easy". "You can do anything". And to do a rev, you didn't need to burn a $50K and 6 weeks (or whatever it is now) at the fab for each turn.

The problem with software is that it is SO unconstrained. You truly CAN do anything - except engineer in an unconstrained environment. I guess this realization (and Python blowing up on me in run-time environments) have taught me that: Constraints suck. But they are good. Software could use more of them, because they force discipline at dev time, at compile time, which reduces blow-ups at run-time.

> People who designed bridges had a wide, but constrained parameter space, and well-accepted design patterns.

And yet they still fall down due to stupid mistakes that nobody noticed.

https://www.usatoday.com/story/news/nation/2019/10/22/design...

Yes, they do. And they make the news.

I was always amazed about the stories of resonant frequency issues, not just from walking but also from wind! - https://en.wikipedia.org/wiki/Tacoma_Narrows_Bridge_(1940)

> I went to an engineering school.

My father attended MIT in the 1940s. He said that the engineers at MIT designed the new chemistry building, with all the latest features to support laboratories. Only when the building was completed and scientists were moving in did anyone realize that there were no toilets.

Ha!

No women - what did they care? :-)

Rumor has it that the U. of Waterloo was designed without including the weight of ... books. So the building is sinking.

This rumor also exists for the Robarts Library at UofT, and in both cases, I am quite certain, it is just that - a rumor.

I think that's why frameworks are so popular: They extensively constrain the solution space. Folks lament their usefulness. ("You don't need React for that! Just use jQuery.", "Why not use raw SQL instead of an ORM?") I don't mean to belittle framework criticisms. Leaky abstractions, over-engineering, and peculiar implementations encouraged by a framework's model are legitimate commonplace warts. We've seemingly converged on these super-APIs because flowing through such a narrow (but not too narrow!) solution space is worth the relatively minor headaches.

Engineering has a long tradition of trial and error. That is how the field of architecture got its current knowledge that currently probably focuses more on material research.

I don't see how software is different. We try new patterns, languages, deployment techniques and think about the optimal way to store data with specific operations in mind. And here you cannot just do anything if you want to solve a problem efficiently.

If we find better practices or more use cases, the environment becomes more restricted. We already have quite a few constrains for a relatively young field because there are many developers.

On the contrary I believe engineering to be a creative process. Of course lacking constrains might make it seem unfocused, but I still call that engineering.

> Engineering has a long tradition of trial and error. That is how the field of architecture got its current knowledge that currently probably focuses more on material research.

Imagine you are building a cathedral and every month the material you use changes to something that behaves totally different. While there is knowledge you can extract even when the material you work with and the fundament you work on seems to change every time, we live in a world that has given up to build systems that last.

In times where deploying a patch meant sending out CDs, software that didn't have to do that was quality software. Nowadays that feeling has flipped: if software doesn't offer updates at least once a month people feel like there is something wrong with it.

To me it feels like we collectively gave up on building things that last in software.

The grand cathedrals in Europe do sort of last, but not without continuing efforts to keep them from collapsing. Nearly everyone I've looked at had centuries of reinforcement of various kinds added.

Strangely, I am on the opposite site - that programming is more of a subset of mathematics (or at least should be) than engineering and since theoretically many parts of the program can be inferred without running, trial and error should not be as acceptable as it is (I don't mean it on a small scale like I do feel sometimes all I do is try different word combinations until the compiler is happy :D).

> Constraints suck. But they are good. Software could use more of them, because they force discipline at dev time, at compile time, which reduces blow-ups at run-time.

Can I take that to mean you're in favour of compile-time enforced strong and static typing?

Not the parent, but that would be very much a yes from me. Writing something safety-critical in a language like Python or JS would terrify me.

If you enjoy having a language which enforces some constraints on you, you may enjoy languages of the ML family.

Elm, for instances, enforces constraints such that the generated js never crashes your browser. Haskell has evolved a powerful system that lets you check web apis against their implementations, which can be pretty handy to enforce that clients should still be able to consume it.

I can recall a few articles where software felt like engineering. bits derived data needs, complexity based hardware sizing. Real solid constraints to use as foundations. rare

People have been building bridges for thousands of years. We've been doing software for barely two generations. Try to imagine what the first generation of bridge-building looked like. Don't worry: we'll catch up. Give us a few thousand years.

The fact that our CPU designs and languages keep changing every year doesn't help. I'm sure there are lots of Z80 experts that know how to develop high quality software for that processor but nowadays we need to support x86, ARM and soon RISC-V. This forces us to use Javascript, Java or Kotlin which are cross platform by default. However, Kotlin is a very new language. Javascript and Java are currently undergoing rapid iteration (Java 14 is out!).

> It is a miracle that the software that drives the world more or less works

It's more of a survivorship bias.

It only takes one bit flip or a single typo to break your program. It only takes a single incorrectly wired transistor in your CPU and it all comes crashing down. Yet somehow it keeps working. The answer is that making decently reliable software is easier than you think.

I don't buy it. I think the real problem is that software became one of the worlds biggest commodities practically overnight and developers are usually more incentivized to get stuff done faster rather than produce higher quality.

There's practically no material cost to software development, the proliferation of the internet makes patching trivial, and the overwhelming majority of software amounts to an nothing more than an inconvenience if it fails. Considering the high value of software and the relatively low cost of bugs, it's only natural that speed is valued so much more than quality.

I'd go even further with this.

I'd say that most software engineers are ultimately powerless to improve software beyond a certain point because doing so would cost more than their employer is willing to spend.

Given enough time, any software engineer could make a reasonably solid product/service that would stand the test of time. Most projects are run on a deficit of either time or resource, and as a result we end up with cut corners.

Ironically it seems people as well as society as a whole fails to account that a zillion times inconvenience is death by a thousand cuts. This includes everyone writing code.

Goddammit when you take a long, hard look, objectively (like really) stepping back for a moment, every piece of software (and hardware to some extent as the line between both is getting blurry) around us is just a huge pile of Rube Goldberg machinery that barely happens to work on the happy path.

This process is not entirely unlike global warming and similar large scale psychological risk assessment failures.

And each individual piece might not be mission critical, but some mission critical component of the system relies on some non-mission critical piece of software or service.

The more I look at how fragile all human created systems are, I realize that humanity will perish with the equivalent of tripping over its shoelaces. Own goal!

New categories of software are created every single day. Things like databases, compilers, filesystems that belong to the well known categories are all rock solid compared to your average iPhone or Android app whose app stores have existed for slightly more than a decade.

The problem I see is that we keep throwing away things that work well and are understood.

Relational databases will work well for most products, its understood how to design them well, optimize them, tune the servers. But nowadays we need to stick everything into microservices on a Kubernetes cluster with Mongo instead and we are told that this is progress.

Reply on news.ycombinator.com