Software should be designed to last

I don't think anyone who is serious about software craftsmanship would dispute the basic idea here ("you should be tasteful and minimalist when choosing dependencies").

The problem is that the economics of software development don't really ever require craftsmanship. There are two basic modes of development. 1 - You are a startup, struggling to survive; you have to ship features as fast as possible, with many fewer engineers than you really need to do things right. 2 - You are a huge billion dollar corporation, probably a monopoly, with an enormous moat protecting you from serious competition.

Neither situation prizes craftsmanship. The startup just needs to glue together a hack to raise the next founding round or solve the immediate user problem. The gigacorp has more money than it knows what to do with, so they "solve" every software problem by hiring more engineers. There's actually not many companies who exist in the middle ground, where it might be important to produce high quality software. I believe this is also the reason we don't see more uptake of high concept languages like Haskell and Lisp.

What this article really boils down to is the usual coder arrogance: Thinking that most problems are inherently simple[1], thinking they are a better coder than average, and thinking that the answer is always to do more yourself, so that more of the codebase will benefit from your superior skills.

The truth is, there are no easy problems in programming. There are always a thousand corner cases and unexpected complexities. And all those buggy, unmaintained libraries you find in your language's package repository are still probably better than anything you're going to roll yourself. Look into the source code of such libraries and alongside the hacks and outdated code, you'll find hundreds of subtle problems avoided, because the author spent a long time and effort on the problem you've spent ten minutes thinking about.

The answer to quality problems in software engineering is not less libraries, it's more and better libraries. In the same way that the high-quality civil engineering the author admires is not achieved by having one person do everything themselves, but by the successful collaboration of a large number of skilled and specialised subcontractors.

Right now, we're still in the infancy of our industry. This, combined with continued the evolution of hardware capabilities has meant that software engineering has never had the long-term, stable base on which to build repeatable, reliable routes to success. That's fine. If we're in the same position in a couple of centuries, it's a problem, but right now we should be experimenting and failing.

Even today, the situation is not as bleak as if often portrayed in articles like this. There are a huge number of robust and high-quality libraries and frameworks providing extremely complex capabilities in an easily reusable way. If a programmer from the mid-nineties were to be transported to the present, I suspect they'd be amazed by what off-the-shelf libraries enable even a single programmer to do. The undeniable quality problems that do affect much modern software often have more to do with the hugely increased scope of ambition in what we expect software to do for us.

[1] Outside a handful, like encryption, that are acknowledged as hard for the purpose of being the exception that proves the rule.

I would like to answer the question "Why is software so frequently disappointing and flawed? "

As a developer, there is nothing I would like better than to turn out the highest quality software, as long as I could be compensated for it.

Does anyone really think that the millions of lines of code that went into Windows is worth the piddling $100+ the consumer pays for it?

Let's start with a basic premise: quality is worth money. I'm sure we agree that a Hyundai (or whatever passes for a cheap car in your neighbourhood) costs less than a Porsche or BMW because the Porsche is better designed and better built. Better design means more experienced and brilliant engineers, more talented designers; better built means more skilled and dedicated assembly-line workers. All these people demand more money. Hence the Porsche company commands a higher price for their Carreras, 911s, whatever.

The same should be true of software. If a company invests great care in designing a better operating system, or a better word processor, that never crash, and always have helpful help and meaningful error messages, how much do you think that would be worth? I'll give you a hint: the military do in fact get top-quality software for their jets and rockets. They get software that almost never fails, and does exactly what it is designed to do.

Do you know how much this software costs? $50 per line of code.

Translating into everyday terms, a bullet-proof operating system would cost you, at a rough guess, $5,000 per copy.

Now I have no doubt some people would be happy to pay $5,000 for a stable OS. However, there are many people who couldn't afford this amount.

So what would happen? In any other field (automobiles, stereos, TVs, restaurant meals, housing) people who can't afford quality just put up with less and shut up.

But in the software field... well, they just make a copy of someone else's software, and enjoy the full benefit of top-of-the-line quality, without paying for it. I wager even you couldn't resist obtaining a $5,000 OS for free.

How long do you think a software company would last if their product cost millions to make, and they only sold a few copies at $5,000? Why they would go broke, of course.

This is the crux of the software dilemma: except in a few specialized cases (commercial or embedded software), the maximum price for software is the monetary equivalent of the nuisance value of duplicating it.

In consumer software, this is in the range of $19-$29.

The digital world turns the economics of quality upside-down: in traditional models where quality is supported by price, the market pays the price if it wants the quality.

In the digital model, a perfect copy of the merchandise costs virtually nothing, and undercuts the legitimate market, putting a cap on the maximum that can be charged for a product.

There is a built-in limit to how much time, effort and expense a company can invest into a mass-produced product. This cap is equivalent to the "nuisance value" defined above. It is not reasonable for the consumer to expect warranties and liabilities that go way beyond what the manufacturer receives from sales of the product.

The music and movie industry are wrestling with the consequences of easy digital duplication. They have taken a different route to protecting their intellectual property.

I challenge anyone to come up with a business model where the software developer that invests great expense in building a quality product, can obtain full compensation from the market segment that values his quality.

Whose fault is it anyway? Simple: it's the consumer who copies and pirates software that forces the price down and therefore the quality to remain low. Any analysis that does not take this into account is simplistic.

It is naïve to think that most developers are not struggling to make ends meet and stay in business (Microsoft notwithstanding).

I want to build software that lasts, but I don't think you really can at this point.

We haven't even figured out fully how we should represent a string or date or number in software let alone have an enduring language or ABI.

I feel like I am building everything on a foundation of sand at it will need to be rebuilt every 5 years, 10 if you're lucky.

I do think it will change, but it will be awhile and by then it will probably just be a few big players making software anyway, kinda like car companies.

The idea that removing external dependencies necessarily simplifies your code and leads to a maintainable system is a fallacy. Use external dependencies appropriately, follow SOLID principles and a clean architecture and you can leverage the benefits of other people’s code without losing the intelligibility of your system.

In the example the author gives, the real mistake is allowing the web framework to define a system that you then try to customise to produce your application. The battle is already lost because your business logic is now a dependency of someone else’s (rotting) generic web app template. This may be appropriate for you if this is a proof of concept or spike but for something with a longer lifetime, any time you save will be paid back with interest when the host framework diverges from your needs.

A JSON parser is not an equivalent class of dependency if you keep it at the periphery of your system. There’s not much point in writing a JSON parser unless you have a particular requirement that cannot be satisfied by a third party library. You should be able to swap it out for a different JSON parser in a few hours.

80:20 seems like a ratio that was just pulled out of the air; I have no data to hand either but for most user applications I would suspect the real ratio is more like 99:1.

Edited to add: I’m still in agreement with a lot of what the author says here, such as: evaluate your dependencies seriously, understand them, and be mindful of their impact on things like binary size.

For truly long lasting software, I feel all code that make up a system should be treated equally; i.e. not divided into "own code" and "third party dependencies". Basically extreme form of dependency vendoring.

But it is very expensive way of doing things, and would not work well with modern constantly updating libraries. But I do personally consider software maintenance to be an antipattern. I'd rather have my software be correct and eventually replaceable than constantly changing and "maintainable"

In order to have software that lasts, you need environments that last.

The classic 1950s Big Iron software, now run swaddled in layer upon layer of emulation on current mainframes, is software which lasts because people will recreate its environment again and again, and ignore the people who are ill-served by it because they don't fit into that environment neatly. Oh, your name doesn't fit into an all-caps fixed-width EBCDIC form? Go fold, spindle, and/or mutilate yourself, NEXT! (This happens to me. Over and over again.)

On the opposite extreme, unmaintained Internet software rots:

https://utcc.utoronto.ca/~cks/space/blog/tech/InternetSoftwa...

Or, to be more precise, software is built with assumptions underpinning it, and those assumptions change out from under it more quickly on the Internet than in other contexts. Software can go from being secure and completely above reproach to being a major factor in DDoS amplification or spam runs because the world changed around it. Software that lasts is like ships that last: Replace the hull, the mast, the sails, the cabins... same ship, neh?

It's very sad for me to see that software in consumer products isn't that reliable or performant anymore, just because vendors can "push updates", which seems to result in crappier software overall.

The kicker these days is that "standing still" is not enough. Software that "lasts" has to constantly change, because externalities force it to - primarily, security.

Whatever library, language or foundation you built on, you can guarantee whatever version you used is going to stop getting security updates in a couple of years. And then there will be a whole lot of baked in dependencies you didn't know about - URLs (like XML schema URIs that were never meant to be resolved but lots of libraries do), certificates, underlying system libraries etc.

So designing software "to last" now is designing software to be constantly updated in a reliable and systematic way. And the interesting thing about that is from what I can observe, the only way to achieve that currently IS through simplicity - as few dependencies as possible, those that you do have very well understood and robust, etc etc. So it sort of comes back to the author's argument in the end, though maybe through a different route.

As a counter point to the author:

* Half baked in-house implementations of things are often filled with bugs and broken edge cases. Especially when it comes to security and concurrency.

* In a larger code base, you won't understand everything whether it's in-house code or third party code. The in-house code was written by someone who quit 5 years ago and is now a monk in Tibet.

Software should be built to accomplish its purpose. It's not a special snowflake, it's a tool to do some specific job. To think otherwise is to setup yourself and your company/team for disaster.

Kernels, compilers, language runtimes, databases? Sure, build them to last. Web pages that will not be used in 2 weeks? Don't waste your time with "build to last".

The problem is that some of us think that our software should be build to last - but in reality it's just some mediocre thing that needs to get in front of the clients as fast as possible and some bugs are "OK" to live with.

I try to use no dependencies beyond my database connection. The one area I feel unprepared to handle myself. Anything else I try to code a minimally-viable functionality that meets my needs. As a bonus, I have learned what all these things do and how they work while fiddling with implementations, and if I want to customize their functionality in any way, it's generally pretty easy.

I'd almost always take a pasted tutorial over a robust dependency every time, the former is more reliable. Any dependency I do take on is something I have to watch the changes of and project status of like a hawk.

Pure HTML with progressive enhancement has worked fine for 30 years and will continue so for many more. But a good, lasting product is incompatible with the current "take the money and run" business strategy and analytics for management. If it works well for so many people, what can we do?

Software is not always designed to last. Take video games that use servers, the video game company can take down the servers and the video game no longer works. You'd have to set up your own server to emulate what the video game company's server did.

All those mobile games that require a server, once the server goes down the game no longer works.

In the old days you set up Doom as a server on your PC networked to other PCs for multiplayer. Those can last and be remade for new platforms.

More often than not, the code I have to write needs to be written fast, and I (mostly) won't have time to look at it again once deployed. I've mostly worked at fast growing companies, so that's been the case everywhere.

What I've learned is that making stuff as simple and stable/low maintenance as possible is the only way to be sane. I can't afford to deploy code that then requires me to hold its hand with a team of 3-4 engineers for the upcoming years. It needs to run itself and be forgettable. And it needs to be written fast.

This has become much harder in recent years with the switch to things like Kubernetes. Kubernetes moves so fast and needs constant nurturing, even for casual users. Running old versions of it is painful and dangerous, so you're forced to update. And the ecosystem is still so early in its life that all the paradigms change and flip on their head every year. Odds are that something you deploy in it today, will need to be looked at in 18 months. And the whole thing will need a team dedicated to keeping it healthy.

Anyhow, that's my angle on the author's rant. Things need to last, IMO, mostly because I don't have time to go back to them later.

If you want your software to last remember that you also have to have a working and upto date development environment, ie your code may still work fine but if you cannot compile it to the correct target it is useless.

I found this out the hard way when I had to recompile some fpga firmware for a $100K piece of equipment which used a programmer not supported by windows, a chip and code not supported by the current IDE and a binary blob which we did not have the source code for and would not link, luckily we winged it by replacing the FPGA with a hardware compatible replacement... We now preserve the whole development environment on a VM and save it with the source code.

If you write your code properly, you will have tests which break on every library incompatibility, and a tiny wrapper around your libraries. (If you do TDD properly, you will have it.) And that's all you need to keep your code up to date. When upgrading a library, you will of course build and test your code with it, and that will show you any problems, which you can fix in the library interface layer before having any chance to go to production.

I think we can all agree that software should be designed to last :)

My experience tells me that when this fails to happen its usually not explicitly due to too many dependencies. Rather it is because the developers have a poor/partial understanding of the problem domain.

External libraries help this problem a bit because it allows developers to offload tasks onto other more experienced developers. They can (and occasionally will) misuse those solutions, but that is not the dependencies faults. On the contrary skilled developers will produce quality solutions with or without dependencies.

The signal seen (poor software has many dependencies) does not mean that software with dependencies are poor. It means instead that developers who have a poor domain understanding but good business skills are disproportionately successful compared to their peers that have good domain understanding but poor business skills.

Make of that what you will :)

Isn't the inevitable conclusion "you should design your interfaces so that they last" (and/or choose existing interfaces which look likely to last) rather than "you should avoid dependencies"?

It's all an illusion. Software kinda isn't lasting, some things merely have competence behind them

Look at the GNU coreutils, things you probably think shouldn't have changed in a while: https://github.com/coreutils/coreutils/tree/master/src

There's actually frequent edits. Legendary disk destroyer was just updated last month. The difference is the competence

Building things to last has a really high cost and is usually not worth it. The vast majority of the buildings and infrastructure humans have constructed have been torn down, completely rebuilt or abandoned. What remains are usually extremely expensive monuments to religon, culture or governmental constructions.

Most software is not worth to preserve the same way most buildings are not worth to preserve (except for perhaps the aesthetically pleasing facade).

"...bridges and civil engineering projects are designed to last, why can we also design software for it to last?"

Some software has lasted for many decades. This isn't necessarily a good thing. [1]

[1] https://arstechnica.com/tech-policy/2020/04/ibm-scrambles-to...

90 days certs that needs renew based on external service API that can change says No.

Software in a hardware that is designed to do certain job should indeed last. Like Telecom towers should work fine "forever".

But software as in APIs and services that constantly has to evolve and be expanded. Yeah that is a harder sell.

We live in crypto world where each advancement in cputing or ways to solve could render algorithms "bad" means we have to update and adapt.

Software that lasts is software that is a joy to use.

Imho the tech, how good you are at programming is all not very relavant when the user doesn't like to use your software.

I have written a small CMS for a company which was replaced twice but every time they ditched it to return to my old simple version.

This is just one example but I think that should be in people's minds when they create software that lasts: people must enjoy using it.

This can even apply to a command line program.

If somebody is a solo developer working on a huge project, it would be lovely if the software lasted... but the big fear is that it won't get users. Can't really spare the time to worry about 10 years from now when you are thinking about how you have to get a regular job if you can't get X users in the next 6 months

Civil engineers and spacecraft designers build their systems to last because they don't have a choice. You can't deploy fixes to a production bridge ten times a day.

There might be good reasons to build software to last, but "These other engineering fields build their things to last" isn't one of them. I have no doubt that if you could reliably send fixes to bridges and spaceships, people would do it - and that many lives would have been saved.

My personal opinion is you should design the larger ecosystem around your software to last and be robust to the replacement of any individual part. Building on Go's standard library HTTP server would have seemed needlessly cutting-edge about five years ago. Building on Apache and mod_cgi would be considered dated today (even though it would work). The stack you should be using will change. Build your system so that, when change comes, you're not afraid to write new software.

In practice, this means making good designs to avoid hairballs of complexity, having deployment pipelines that let you roll out canary versions (or better yet, send a copy of production traffic against a sandbox environment), keeping track of who's using APIs internally so you can get them to change, having pre-push integration testing per the Not Rocket Science Rule so you can be confident about changes, etc. You should think about your choice of web framework or JSON library up front, sure, but more importantly, if a sufficiently better web framework or JSON library comes along tomorrow, you should be able to switch. As a principle - any time you see something that you think you'd be afraid to change in the future, don't ship it if you haven't shipped it yet and figure out how to build the right abstraction around it if you can.

Reusable abstractions is the foundation of what makes programming productive. This person does give respect for libp2p, which is good, but clearly they've never seen a small abstraction that is still immensely valuable.

The software (as in the actual code for an implementation), should be ephemeral in my opinion.

The data model, protocols and knowledge base (docs, requirements etc) should be designed to last and be extensible.

Constructive criticism if the author is in here, I find multiple bold sentences per paragraph to be taxing on the eyes.

> If all three are in agreement, they carry out the operation. If even a single command is in disagreement, the controller carries out the command from the processor which had previously been sending the correct commands.

By this definition, wouldn't all three processes be the last ones to send the correct commands?

for some reason when I saw the headline I went over to check up on grc.com that guy's obsessiveness into coding everything in assembly gives me comfort

>> Software should be designed to last

Not necessarily true.

If the requirements of the project are to be secure or fast or easy to maintain or built quickly or have beautiful user experience or be maintainable or be designed to be robust & failure resistant, then that is what it should be.

And if the requirements of the project are "the software must be designed to last", then the software should be designed to be designed to last.

But if it's not a requirement, then no, it does not matter if it lasts.

The point being that blanket statements about what software should be are missing important context about the purpose of the software and the constraints under which it was developed and the specified requirements.

This is why I like long-established technologies. You can build most anything with typical POSIX utilities and time-worn libraries. And they will be there for you in 25 years. Who knows where Rust and Go and Node will be in a quarter century?

no, software should be designed to work for as long as necessary, because we're responsible professional and don't want to squander other people money on infinite perfection

the whole bit is nonsensical included the parallels with civil engineering, bridges are built with tolerances, maintenance schedule, and a disposal plan for when maintenance cost become higher than replacing the bridge, because nothing in this world actually lasts.

heck our internet is built around operating systems that didn't exist 25 years ago. imagine investing capital and sink opportunity costs in delivering the perfect vax typing program...

mind, this is not to say that all software should be quick and dirty. but the conscious engineer knows whether it's building infrastructure to last decades or a temporary bypass to support some time limited traffic spike.

the important bit is to ask the stakeholder what's more appropriate.

I also want to see people build their own json parsers at acceptable level of performance, possibly as streaming parsers instead of just gobbling everything in memory. imagine wasting a month on building that and then having to defend your choice every time a bug or a malformed input causes silent data corruption somewhere in the backend. http://seriot.ch/parsing_json.php