> software today is endemically crappy, and often poorly designed
So is every other engineering product that hasn't gone through years (decades) of evolutionary refinement.
So is every other engineering product that hasn't gone through years (decades) of evolutionary refinement.
I went to an engineering school. People who designed bridges had a wide, but constrained parameter space, and well-accepted design patterns.
I started out in semiconductor (MCU) systems and sometimes circuit design, and we had a broader (but not yuuuge!) parameter space, but it was growing as transistors got cheaper. Less well-accepted design patterns, because what you do with 50K transistors and 500K transistors and 5M transistors and 50M transistors - to use effectively - you need different patterns - and that changed so fast!
I did software-ish things with my h/w teams, and they would mock me because "software is easy". "You can do anything". And to do a rev, you didn't need to burn a $50K and 6 weeks (or whatever it is now) at the fab for each turn.
The problem with software is that it is SO unconstrained. You truly CAN do anything - except engineer in an unconstrained environment. I guess this realization (and Python blowing up on me in run-time environments) have taught me that: Constraints suck. But they are good. Software could use more of them, because they force discipline at dev time, at compile time, which reduces blow-ups at run-time.
And yet they still fall down due to stupid mistakes that nobody noticed.
https://www.usatoday.com/story/news/nation/2019/10/22/design...
I was always amazed about the stories of resonant frequency issues, not just from walking but also from wind! - https://en.wikipedia.org/wiki/Tacoma_Narrows_Bridge_(1940)
My father attended MIT in the 1940s. He said that the engineers at MIT designed the new chemistry building, with all the latest features to support laboratories. Only when the building was completed and scientists were moving in did anyone realize that there were no toilets.
No women - what did they care? :-)
I don't see how software is different. We try new patterns, languages, deployment techniques and think about the optimal way to store data with specific operations in mind. And here you cannot just do anything if you want to solve a problem efficiently.
If we find better practices or more use cases, the environment becomes more restricted. We already have quite a few constrains for a relatively young field because there are many developers.
On the contrary I believe engineering to be a creative process. Of course lacking constrains might make it seem unfocused, but I still call that engineering.
Imagine you are building a cathedral and every month the material you use changes to something that behaves totally different. While there is knowledge you can extract even when the material you work with and the fundament you work on seems to change every time, we live in a world that has given up to build systems that last.
In times where deploying a patch meant sending out CDs, software that didn't have to do that was quality software. Nowadays that feeling has flipped: if software doesn't offer updates at least once a month people feel like there is something wrong with it.
To me it feels like we collectively gave up on building things that last in software.
Can I take that to mean you're in favour of compile-time enforced strong and static typing?
Elm, for instances, enforces constraints such that the generated js never crashes your browser. Haskell has evolved a powerful system that lets you check web apis against their implementations, which can be pretty handy to enforce that clients should still be able to consume it.
It's more of a survivorship bias.
There's practically no material cost to software development, the proliferation of the internet makes patching trivial, and the overwhelming majority of software amounts to an nothing more than an inconvenience if it fails. Considering the high value of software and the relatively low cost of bugs, it's only natural that speed is valued so much more than quality.
I'd say that most software engineers are ultimately powerless to improve software beyond a certain point because doing so would cost more than their employer is willing to spend.
Given enough time, any software engineer could make a reasonably solid product/service that would stand the test of time. Most projects are run on a deficit of either time or resource, and as a result we end up with cut corners.
Goddammit when you take a long, hard look, objectively (like really) stepping back for a moment, every piece of software (and hardware to some extent as the line between both is getting blurry) around us is just a huge pile of Rube Goldberg machinery that barely happens to work on the happy path.
This process is not entirely unlike global warming and similar large scale psychological risk assessment failures.
The more I look at how fragile all human created systems are, I realize that humanity will perish with the equivalent of tripping over its shoelaces. Own goal!
Relational databases will work well for most products, its understood how to design them well, optimize them, tune the servers. But nowadays we need to stick everything into microservices on a Kubernetes cluster with Mongo instead and we are told that this is progress.
My personal recent experience of actually doing software the old-fashioned way involved writing correctness critical high-performance code for a major data warehouse vendor, which mostly stayed with zero known bugs in production - greatly contributing to the vendor's reputation as reliable and dependable place to keep your data in.
And how do I know what the old-fashioned way to write code is? Well, I've been writing code professionally for nearly 40 years.
If you consider why a company makes something reliable or not, it's a relatively simple formula:
= Number of Users x (Benefit of Getting it Right x Probability of Getting it Right) - (Cost of Getting it Wrong x (1 - Probability of Getting it Right)))
[talking about non-software engineering] There's a huge difference between reading books about proper engineering and how it is actually practiced. Much of the sloppiness is covered by simply over-engineering it. Designs are constantly improved over time based on service experience. Heck, lots of times the first machine off the line cannot even be assembled because the design dimensions on the drawings are wrong.
The idea that non-software engineering is done by careful professionals following best practice and not making lots of mistakes is painfully wrong.
But then, non-software engineering is a wide field. There are products that you can afford to iterate on, and then there are very expensive (and potentially very dangerous) projects where you generally can't afford many slip ups.
If your engineers make lots of mistakes (which aren't caught in time) in a project that costs millions and can't be replaced by another unit off an assembly line, that's kind of a big deal. Thankfully, we don't hear about bridges, skyscrapers, heavy industrial lifts or nuclear power plants failing all that often.
The first nuke plants are pretty dangerous by modern standards, and we know how to fix them, but because the first ones are dangerous we are not allowed to build fixed ones.
The Fukushima plant, for example, had numerous engineering faults leading to the accident that would be easily and inexpensively corrected if we were iterating.
Airplanes are a good example of how good things can get if you're allowed to iterate and fix the mistakes.
You mean all that softwarte written in '90s with no security in mind?
I bet you're talking about outliers. Nowadays average developer practices are levels above that what was decades ago while being supported by great tooling.
You can have an extremely reliable piece of software running say, an industrial lathe or a stamper or book printer or whatever - software which can run 24/7 for years if not decades, software which will never leak memory, enter some unknown state or put anyone in harms way - and yet have zero "security", because if you plug in a usb keyboard you can just change whatever and break it entirely. Software which has no user authentication of any kind, because if you are on the factory floor that means you already have access anyway because the authorization step happens elsewhere(at employee gates etc).
It's like people making fun out of old ATMs still running Windows XP, because it's "not secure". If the machine isn't connected to the internet, reliability is far more important - who cares windows XP is not "secure" if the ATM can run constantly for years and reliably dispense money as instructed and there isn't a remote way to exploit it.
I feel like that first kind of software(the reliable kind) is far rarer today - people just throw together a few python packages and rely on them being stable, without any kind of deeper understanding of how the system actually works, and they call themselves software engineers. The "security" part usually also comes as a side effect of using libraries or tools which are just built with "security" in mind, but without deeper understanding what having truly secure software entails.