Software developers have stopped caring about reliability | Hacker News Reader

Software developers have stopped caring about reliability

drewdevault.com

Well, it's easy to complain.

The reality is that money dictates decisions and behaviour; and if consumers wanted reliable stuff, they'd buy it.

They don't.

They want free stuff, and if it doesn't work well, they'll complain and suck it up because it's free.

The idea that developers are somehow empowered to a) not do as they're told by their bosses, and b) should just go off on their own and do whatever the hell they want (eg. making things reliable), when it is not what they have been asked or instructed to do...

...well, it's a big ask.

If you want change you need to do one of probably these things:

- Empower developers to somehow write more reliable software at no additional effort.

- Convince consumers to buy more expensive stuff.

- Provide a legal mandate to punish companies that fail to adhere to specific levels of quality control.

- Provide a meta framework the punish individuals that fail to adhere to specific levels of quality control (eg. IEEE and other professional organisations which can expel you).

I can't imagine how you do the first two. Those are Hard Problems.

The last two have worked in the past in some circumstances in other professions... but, you know, many folk don't like the idea that there are codes of behaviour, ethics and conduct they should be forced to adhere to; but they want people to take personal responsibility for things not being perfect.

It's a bit of a joke that everyone has to 'do the right thing' with no rules to enforce it... but hey, it's easy to ask for, because you don't have to do anything except complain people aren't doing the right thing.

Thats why you can't have nice things like software that works.

> Provide a legal mandate to punish companies that fail to adhere to specific levels of quality control.

This is the only workable solution - fortunately it fits right into established regulatory frameworks.

-------------

Controversial opinion:

I do think we need some kind of widely-recognized (i.e. not just in Texas) professional licensing system for SEs/SWEs to enable some kind of accountability for bad software products:

...When a building collapses and kills people, the civil engineer-of-record is accountable or serve as witness. Similarly when avionics software kills passengers (MCAS, etc) there should be a software-engineer-of-record to be held accountable or to serve as witness. Change my opinion.

I know there's a big difference between safety-critical software systems and consumer-hostile inkjet printer cartridges ( https://news.ycombinator.com/item?id=28888214 ) - but more and more jurisdictions (especially outside the US) are going to call-out US and international software and hardware companies on their business-practices which means they'll probably institute something like this inevitably.

The difference is that construction is largely doing the exact same thing that has been refined over thousands of years and we know very well how it goes and have formulas to precalculate everything. So you can have a set of building codes which if you follow, are very likely to produce a perfectly fine building.

While software development is completely making things up as you go because it hasn’t ever been done before. Any kind of government regulation would end up as a box checking exercise that wouldn’t add any more safety and probably introduce problems when products have nonsensical decisions added in to fit with some requirements.

On the other hand our ability to precalculate everything in construction seems to be fairly novel, in the sense that complex geometries are very hard to calculate without computer simulations. And most buildings we build today are nothing like those we built 200 years ago.

The necessity of modelling building behaviour has pushed the field forwards. For software we know that modelling and verification works in principle, but little time is spent on improving the methods because there's no demand

I counter that with that we’ve got methods of developing relyable software and we’ve got examples of success. We’ve got all sorts of different tiers of testing and verification, we’ve got various CI/CD approaches and NASA seems to be landing one rover after another and flying an freaking helicopter on a freaking another planet, autonomously.

We’ve got the tools to explore unknowns.

Now, as to regulation. It could be written in a way that would relieve that “developer-of-record” out of responcibility provided they did their due diligence and, if new unknown happened that wasn’t foreseen the regulations will adapt and change and make a learning out of it, just like it was with building industry and road rules.

> I counter that with that we’ve got methods of developing relyable software

We don't have methods for reliable spelling, however...

My spelling is horrible, especially when I write on the phone, I know, sorry

You're fine :) it's just ironic

I'll add that we expect software to work on highly heterogeneous systems, too.

If you're working on a bridge, you known exactly the terrain you're required to span. The engineers battle with a lot of integration issues, but at least they know what they're dealing with.

With software development, well, who's to say what integration issues that millions of people might face?

> specific levels of quality control.

Good luck defining that though.

The main difference between programming and engineering is unfortunately that no one wants you to build the same thing over and over. So you have no idea if it will work until you build it.

Please don't point me to formal correctness proofs, they just won't work at scale. Hmm, perhaps some bright mind will come with a proof that correctness proofs are too complex to be practical.

> The main difference between programming and engineering is unfortunately that no one wants you to build the same thing over and over. So you have no idea if it will work until you build it.

In between there is the radical idea of writing code carefully, testing, and documenting it before shipping it. All of us as programmers want to do that, but we have all worked under management that insisted on the opposite.

Define that too :) Budget restrictions aside.

For some young things I'm working with, basic debugging techniques seem to be foreign as well...

The CII Best Practices initiative sets a good set of requirements to qualify. Mission-critical software needs to have a good bus factor, keep its deps up to date, have guards in place for the OWASP Top 10 vulns and others (where appropriate), not lean heavily on badness enumeration, etc.

We can set requirements for certain types of software to get certified for a level of reliability, taking inspiration from the CII Best Practices. Two ways to qualify for a certification can include testing and formal verification; I won't elaborate on the latter as per your request.

Good statement coverage from tests is a necessary but absolutely insufficient requirement for tests; they need good branch coverage, and need to pass a good mutation score threshold. Property-based testing makes this a lot easier.

Fuzz-testing is another good follow-up once these get checked off.

Software that passes the above checks and more can qualify for manual review, looking for common vulns and warranted practices.

Much of what I've brainstormed above applies to complex software; simple/small software is much more manageable and probably doesn't need to be held to such standards.

Is the Texas Software PE exam still there? I looked into it years ago out of curiosity, but now I’m seeing that it might not be around any more[0]

> NCEES discontinued the PE Software exam after April 2019.

I think the industry would benefit from an FE style exam at the very least.

[0] https://ncees.org/engineering/pe/software/

I agree with you. It is time. Code runs too much of our world to not have a formalized system of accountability. At least in some domains to start.

One thing though, in the US, and in the case of a building, it would be a Structural Engineer not Civil. And specificity a PE, or Professional Engineer who has a personalized rubber stamp mark registered with the state.

At least that’s what I recall from supporting a CAD department 25 years ago.

I generally agree with you, but where do you draw the line on what software needs this accountability? Particularly when things change pretty quickly in the field. As an example, 30 years ago, a lot of the internet was not 'important' yet it grew to critical for most of our systems.

In my opinion, your comment and all the comments replying to it (as of writing this comment) are well-considered arguments with valid points.

To summarise:

- we need a licensing system because it promotes accountability and encourages companies to think long-term even when the short-term benefits are tantalising.

- but everything we do in SWE is so new that any licensing system run the risk of (a) being too loose so as to be easily gamed or ignored (think cookie banners a la GDPR), or (b) being too labyrinthian that it kills innovation

Being a product designer, I am directly affected by many of the same issues as engineers. I would be happy to indulge in some level-headed debate. I will be watching this thread so please reply if you have any insight to add! Thank you.

"Show me the incentives and I will show you the outcome." ~ Charlie Munger

> if consumers wanted reliable stuff, they'd buy it.

Trouble is, often I can’t when I want to because no one’s making it—that the industry (whatever industry, you find this in many of them) has accepted mediocrity.

For software specifically, my observation is that attempts to retrofit reliability, performance and the likes almost always fail: these are properties that you have to deliberately bake in with from the start, and maintain lest you find them difficult to claw back.

> The idea that developers are somehow empowered to a) not do as they're told by their bosses, and b) should just go off on their own and do whatever the hell they want (eg. making things reliable), when it is not what they have been asked or instructed to do... > > ...well, it's a big ask.

While back my brother's band had a song called, Money Goes to the Man. It captured this issue pretty well.

It seems much easier to find examples of the first two than the last two, really. I've been programming now for 30 years. Here are just ten of the major improvements the industry made to software reliability in that time, on its own, without government intervention of any kind:

1. Unit testing.

2. Garbage collection (no more use-after-free bugs).

3. Stronger type systems.

4. Then stronger type systems with usability close to that of dynamic type systems.

5. Widespread usage of encryption, signing and sandboxing (hacking being a subset of "stuff that makes software unreliable").

6. Exceptions with useful stack traces, that in some cases can be caught allowing the program to proceed. For example, my IDE doesn't totally die if a plugin hits an assertion whilst analyzing code, it just means that plugin doesn't get to contribute to my editor session.

7. Tightly specified memory models like the JMM.

8. Excellent monitoring platforms.

9. State machine replication like Raft, allowing for entire datacenter outages to be made nearly invisible to end users. Result: Google servers are now more reliable than the internet itself.

10. Ultra-stable operating systems like Linux. In 1995 the most popular OS couldn't manage an uptime of more than ~45 days. Today you can hotpatch the kernel without a reboot.

There are loads more like this. Software today is drastically more reliable than it once was. Meanwhile, what has the government done? I'm trying to think of some, but they've all been failures or actually made things worse. In the security realm, we can thank governments for things like FIPS cryptography, which is hardly relevant outside places where it's mandated because it takes so long to approve what are clearly upgrades. Governments can't even get reliable software out of their own contractors - they're in no position to lecture others on how to do it right when they can't even get their own house in order.

But it's not enough to only consider what governments do for reliability. We must also consider what they do against it. There unfortunately the karmic balance is deeply in the red, because governments routinely stockpile exploits then lose control of them, undermine cryptographic standards and so on. They actually like software being unreliable because they see bugs as weapons. A big part of Google's Project Zero is about imposing software reliability against the will of governments.

In some industries where the consequences of failing software are fatal, e.g. medical devices or vehicles, 3. and 4. are in place and ensure that (usually) nothing bad happens.

2. isn't that important if everyone within an industry has to adhere to the same standards.

For example, GDPR levels the playing field for any company that operates in Europe and ISO27001 etc for many industries. But sure, companies will usually do the minimum required effort - and that only if the threat of fines and audits is high enough.

Ah I remember when I thought like this. I was so young and naive. Good times.

Fact of the matter is, software is NOT like building a bridge, or any other engineering discipline, it's much, much more complex. We'll never achieve high reliability for anything but the simplest software (reactor control rooms come to mind). Accept the fact that software is hopelessly broken, and find tactics to deal with it.

I pretty much agree with this and I also think the reality is people don't care. The stakes are so low that there's no money in well-built software (most of the time). The average user isn't paying out of their pocket and wouldn't even if you offered a more stable experience in exchange for cash.

Agree and disagree. On the one hand, I do agree a lot with the idea that software is a moving target, and it's not enough to build, but you have to maintain it. The whole ecosystem is continually changing, needs are changing, best ways to do things are changing. Formats change, protocols change, etc. I believe acknowledging that is vital in being able to become an effective software developer (I mean, if you are working with a microcontroller or a very well defined and fixed environment, it's another story, of course).

But on the other hand, I don't want to accept that we will never achieve high reliability. I've heard and read multiple times that software goes through an expansion phase (an exploratory phase), until we eventually manage to gain enough insight and clarity as to be able to go back and simplify to something that we can wrap our head around again. Certainly, while developing, I often find myself in this situation. For many problems, you don't understand them until you have written the solution, come across new problems, and are able to rewrite again with much greater understanding... a clean solution.

Now, there are many very complex problems. Some solutions will never be clean. But we do definitely have light years of room for improvement. Networking is messy. OpenGL / Vulkan / low level graphic APIs are unwieldy and inelegant. Same for most databases and SQL. Audio APIs are not much more pleasant. Real time systems are mythical creatures for the average developer. Filesystems are not as homogeneous and standardized as we would like. Browsers are monsters. We can't even agree on what units use for UI elements. We don't really have much idea of how to handle screens of different sizes. Common file formats shouldn't even be a software concern most of the time. We talk so much about type systems but still define most protocol specs in plain text. Interoperability is mostly C or die. Unicode is many things besides a standard for symbols. Too many. Permissions and privacy are... well, maybe it's better to stop for today.

And that's indeed the reason why software must be a moving target. Because we can't accept these foundations to stay as our original, unerasable sin. But I do believe there is a real possibility to do better. One day devs will unite in a mystical place outside the realm of "pressure to deliver", we will agree on what are the main priorities and opportunities for improvement, and we will coordinate to start changing things and make our asylum a saner place. Coordination might be both the biggest technical and non-technical problem we will ever have to face, but I find no reason to believe we can't handle it.

> One day devs will unite in a mystical place outside the realm of "pressure to deliver", we will agree on what are the main priorities and opportunities for improvement, and we will coordinate to start changing things and make our asylum a saner place.

I believe this is exactly what free software is - and in a distant future it will replace all proprietary solutions due to its higher quality.

Vulkan isn't really messy, it's just a very low level API. The idea is that the OpenGL middle ground stopped mattering a long time ago. If you want to just render some 3D objects then you should use a 3D engine, if you want to write a 3D engine then you want Vulkan.

Agree with the rest though. From time to time, the industry manages to go back to the drawing board and make big simplifications or reliability upgrades, quantum leaps. Java was one example because it combined garbage collection, a somewhat rigorous language, high performance, a clean reset of the toolchain (no messy autoconf/Makefiles) and very robust/well engineered standard library. It was a huge success for that reason: familiar enough to be easy to pick up, but it abstracted or eliminated vast complexity.

SystemD was a smaller scale example of a Big Simplifying Reset in the UNIX world.

But these sorts of things come along rarely, and entropic decay into ever more complexity never stops.

IIUC NASA achieve high reliability for problems that don’t seem simple.

Yeah, because their incentives strongly align with it. The argument is that the incentives don't align outside of mission critical systems.

Right, agreed. I was replying to this from the comment. Which seemed to claim it was impossible.

> We'll never achieve high reliability for anything but the simplest software

Sorry, I wasn’t very clear!

Sometimes your seemingly non mission critical software can be exploited in such a way that the entire system is cracked wide open.

True and unfortunately this doesn't often result in any damage until way later when someone buys your sensitive documents and commits fraud. By the time the fraud happens, we likely have no idea which breach the info even came from.

That’s a good rant but it hits the wrong dudes. We haven’t stopped, we were overwhelmed.

Our industry is characterized by institutional recklessness and a callous lack of empathy for our users. It’s time for a come-to-jesus moment. This is our fault, and yes, dear reader, you are included in that statement. We are personally responsible for this disaster, and we must do our part to correct it.

No, the examples the author provides are the fault of stubborn people who continue to say that a winword-like text renderer with left-right-justify-float-border-margin is the right tool for applications. There is javascript in the button because otherwise there was javascript at the serverside, or even more javascript to reload the current state into the new tab. Controls have javascript in them because native behavior is so full of bullshit. And why it’s javascript? Because we’re given no choice.

I tried to bring up the topic of “web” “applications” so many times only to hear that I’m wrong and I don’t get it. While having around 15 years of desktop development experience and a couple of ui frameworks that I wrote myself. And clearly seeing the insanity of modern state of things and what they gave birth to. Back in the day I was able to hack together gta:sa vehicle editor or a simple accounting system in pygtk in an hour, now I don’t even know if I want to begin to map state to props and do other bs unrelated to the business itself.

This is what you must do. You must prioritize simplicity. You and I are not smart enough to be clever, so don’t try.

I stopped trying already, let this crap burn in flames and maybe I will still be not dead when the springtime comes.

Design your data, identify error classes

It’s hard to do through layers of what you need to be able build a decent app, but we try hard, really. The author comes from the web1 perspective and feels missing the simplicity of it, but bad news it was already a stupid way to make apps even back then.

Well to be fair I think Drew and I are cut from the same cloth. We write software tons of people use every day but almost none of it is for the web (at least that isn't the core interface generally).

The whole web2 and now web3 stuff mostly passed us by and now we are looking around trying to work out what went so awfully wrong with software development.

The reality is that guys like us still write highly reliable software, some of it even powers the backends of these awfully buggy frontends people rant about.

But you hit the nail on the head. We got outnumbered, overwhelmed and too tired to put up the good fight every day.

When your coworkers want to build the latest app on Node.js + AWS lambda + other hipster nonsense there isn't much you can do other than say "I told you so" when you are the one stuck migrating and cleaning it up 6-12 months later onto something actually reasonable.

You could say that we should do a better job at hiring but often times that is out of our hands, the company hires what is available and increasingly often that is green engineers that don't want to learn old school reliable stuff like Java/C#, they want to be using blog-post/talk-worthy tech stacks and they haven't yet gained the experience that teaches you why that is a poor idea.

So yeah, senior devs that care are out there but we are tired and we aren't able to fix the system from the inside sadly.

We write software tons of people use every day but almost none of it is for the web

Oh, I had a strong impression from the article that he was a classical-web guy aghasted by new-web, my apologies then.

Heh I think the contents of the post excuses you for making that assumption. :)

For context he is mostly known for his work around Wayland/Sway but these days he also works on sr.ht which does have a web frontend but it's primary interface is the git protocol.

I heard an anecdote about when Unix was being written. They decided that they needed an editor, and figured it would take one person a week to do it.

Clearly, there was a whole different mindset then than there is now.

If they had dedicated another two weeks to the editor that might have spared us decades of people asking how to exit vi(m).

Wow, Brendan Eich is supposed to have had a whole 10 days to come up with JavaScript.

Maybe the managers and product owners decided it is not a priority for this sprint as it does not fit this quartals OKRs?

Maybe we should not work for this kind of place.

But these project members are the same which hire fresh out of boot camp developers and not enough QA. And if they hire QA then QA just checks features, not “is this actual correct” or “does the UI actually empower the users”.

The one consumer software I really hate right now (other than MS Teams and Oracle HCM) is the Amazon Music app. I like the design, but find so many small bugs there. I’m the customer, not their QA or product manager. But if I want to use a better product I either use another app or write them what they got wrong.

Maybe 75% of us would like to be jobless? I dunno, doesn’t seem very likely.

I was there back in the 90s and they cared even less back then. If anything, things have improved over time, I don't even know what the Windows 10 crash screen looks like.

THIS.

The quality of software I can churn out on a hungover weekend far exceeds what I could realistically produce in a professional week, using terrible 90s tools, even with my skills advances since then.

I dont agree with OP. In fact on the contrary, much of the backend software is becoming lot more reliable than it ever was before. Lately most of the back end dev is leveraging various cloud services like dynamodb, big table etc, which have north of 99.99% uptime. Back when I was at Yahoo!, reaching those SLIs was a monumental goal. But nowadays its expected.

Any microservice built leveraging these cloud services, is already far more reliable than building things from scratch. Obviously, there is lots of room to shoot oneself in the foot. But overall, I am far more optimistic of backend core services than what OP presents.

I agree. The author seems to be hung up on the way things used to be but the market shifted completely.

Containers completely changed the market and made redundancy cheap. As a result you might have a fault but overall the system will be far more stable. Staged rollouts used to be reserved for top tier enterprises. They are now accessible to everyone.

His main complaint is about clients but I think his memory of the good old days is seriously lacking or tainted. Software used to suck and web based software has a huge boost in stability and reliability.

The problem is you're both right. Containers worked great to isolate and manage a large class of faults (though arguably e.g. BEAM would handle the same class of faults better).

Let's say fault chance due to program design was x and container orchestration fails also to correct an independent y, so we went from reliability (1 - x) to (1 - x * y). Total system reliability goes up even thought we didn't get better at writing programs, though so does complexity.

But actually, the market generally only wants (1 - x) reliability. That y was invented for the few situations situations where x alone was unacceptably large relative to other classes of software. Now everyone is using it, which means x drops proportionately across the board. If container orchestration made you 10x as reliable, most CFOs and customers will be even happier with 2x as reliable at the same cost, so your actual software will degrade to be 5x more unreliable.

So for people who want to use "software" and not "systems" (which includes developers, hobbyists, and many people in specific lines of technical work who combine prêt-à-porter with bespoke software) the world now sucks horribly.

Agreed. TL;DR: Application-level reliability did not improve. Orchestration/Ops System-level reliability made up for it.

These managed services are of less benefit when the websites that are running on them, like the authors describe, are flaky and unreliable. Like the OP I open the js debugger often - including for relatives.

There have been quite a few major Internet disruptions in recent times and all those "lot more reliable" services proved to not be that reliable after all. We're talking about hours of downtime here. And all those who decided to depend on them were suddenly down, at the mercy of their overlords.

It may get better over time but so far it hasn't exactly been a panacea.

reliability is giving the right answer. availability is being ready to give the right answer.

cloud services have poor reliability and good availability.

I'd love to work on reliability. Unfortunately, my TODO list contains months of new features that need to be shipped, asap. My clients push aggressively to get the new stuff done, reliability almost never something they're interested in, as long as it is not embarrassingly obviously broken feature.

I do what I can, given the constraints imposed on me (since I also hate buggy software), but it's very tough when it's not a key customer priority.

,,Finally, stop letting economics decide everything you do.''

Money is one of the best inventions of humans, and a great organizer in the world. I would say a better organizer than the writer of this article. It's not perfect (especially nowdays with 0% interest rates), but still a good guidance on what value the customer wants.

What about ads? "still a good guidance on what value the customer wants" or is it just blatant manipulation to coerce the customer into buying what you want?

The biggest upgrade to ads in decades was targeted text ads, which aren't manipulative in any way. They barely have enough space to explain what they're advertising, let alone "manipulate" people. And those ads power the world's most successful tech companies because yep, they provide amazing guidance on what the customer wants!

It would be much easier to enforce ethical guidelines for ads and stop the coercion than to enforce software trustworthiness.

Yes. What the consumer wants is a free product over a better one that costs money. HN is probably one of the only places where that isn't a majority view. "Manipulation" is an excuse by those of us with a minority opinion for why things should be done our way rather than what the majority wants.

In the ads business the ad buyer is the customer, not the user. For people to be able to pay in small amounts, I believe that the lightning network is a great solution in the future, but right now either the user pays by looking at ads, or by monthly subscription, like Netflix.

While I agree with much of this, it also reeks of confirmation bias. It's very difficult to notice when software is reliable, but poor reliability can often (but not always) be quite apparent.

I feel this is taking extremely rose-tinted view of past. I don't think software was ever particularly reliable.

Some of it was. My first three computers never crashed at all.

But then hard drives were invented, and suddenly computers could hold several orders of magnitude more mistakes. Demands on feature set grew. We started expecting to run more than one program at once and have them play nice with each other. Then we connected them to the Internet and a whole new world of mistakes became possible - and this is where we are.

But stuff like ... Borland TurboPascal. Was rock solid and fast.

Eh. Turbo Pascal was great but I recall the Delphi era definitely having a tick/tock cadence to reliability. Delphi 2 was solid. Delphi 3 crashed a lot. Delphi 4 was better but still didn't reach Delphi 2 levels of reliability. IIRC 5 was a hot mess. That's about the time I stopped using it.

Yeah, that was exactly what I was thinking. Getting PC games to run in the 90s was an engineering discipline in and of itself.

That's because the PC platform was always a broken mess (and one of my pet theories is that this "mindset" of a broken platform has leaked into the software developed for PCs).

Running games on any non-PC machine (like the Amiga or Atari ST) generally didn't require any tinkering.

Of course this "broken mess" also led to a higher level of innovation in the PC world which eventually left the Amigas and Ataris behind, basically the original "move fast and break things" platform ;)

Oh yes! I had forgotten the tiresome collection of labelled boot disks, and the trial and error to create a combination which would work for a particular game.

Do I skipping loading the CD-ROM drivers for this one, or the sound card drivers?

Do I want extended memory or expanded memory or both or neither?

> I don't think software was ever particularly reliable

Agreed - but software was also markedly less user-hostile and performance-wasteful: though that was due to limitations of the time: e.g. you can't spy on users or require phone-home DRM without a good internet connection (not that DRM dongles were much better).

If there's one good thing that might come from the trend towards computers locking users out of kernel-mode it's that games and media systems will stop being able to install their own rootkits.

DRM dongles were horrible bastards. I remember working with an expensive piece of audio post production software that held it's license and the license for all the plugins on this stupendously delicate USB thing. People plugged them into the front of their Macs and snapped them off so often.

> People plugged them into the front of their Macs and snapped them off so often.

I winced

Ah. A good old-fashioned Jerry Maguire rant.

While I totally feel for the author, as someone else commented, money talks. People pay for dross, and companies will build their infrastructure around creating and shipping dross.

For myself, I am totally focused on writing and shipping ultra-high-Quality software. It’s difficult, frustrating, time-consuming, and very, very rewarding (but mostly on an emotional level).

I work for free. Honestly, I can’t see commercial entities paying the very substantial premium required to ship software at the Quality bar that I set. It’s orders of magnitude costlier than junk-grade software. Just yesterday, I spent almost all day, “tuning” a popover presentation controller to manage and present flexible content within a 1-pixel tolerance. It required that I rip out almost all my previous work, and realign my fundamental design.

I can think of no corporate entity that would tolerate that kind of anal-retentive level of detail. I would have been forced to ship the crap I had working the day before. People really, really like using my software, but they also don’t pay for it.

Calls for licensing are quite reasonable for things like medical devices, transportation, educational, infrastructure, children’s software, etc., but would be ridiculous for a significant swath of SV’s products.

As someone who has worked in a highly-structured, Quality-focused environment, I can report that the structure can easily crush innovation and development velocity. I call it “concrete galoshes”[0].

I feel that the industry is currently run by folks that use money as the only measure of success, and they aren’t actually mistaken. Money brings power and influence. It drives culture. The people who make the money, make the rules (the classic “golden rule”).

Until a reward structure (as opposed to a regulatory structure) is in place to encourage (as opposed to enforce) high Quality, I suspect we won’t be seeing significant improvement.

[0] https://littlegreenviper.com/miscellany/concrete-galoshes/

This can be said about pretty much any field these days. Graphic design is shit, product design sucks, home interiors are all the same with chinese made single purpose kitchen tools, unrepairable gadgets surround you, iot apps last updated in 2016, complete assault on user-centric design, broken touchscreen displays in cars and feature creep driven software that will eventually become unusable. No one cares. Everything is fine. Nod along, pour a glass and sip some plastic bottle moonshine.

"Market efficiencies" have led to a drop in quality. All items/services have become "consumable".

The one concrete example he uses is a broken form to pay for electricity.

I’m pretty sure that the form works for most people most of the time. Why? Because otherwise the electricity utility in question would be hemorhaging money. If it would be completely borked the beancounters would be kicking the developers door in in no time.

I cannot say if this is what is happening but I have seen this many times where an expert user gets hung up on some process because they choose options only an expert user would even know about, while masses of regular people passes through uneventfully.

Sometimes all it takes is an adblocker that breaks some tracking functionality that interacts with the payment form. An event handler throws, and the whole thing falls apart.

Sometimes it’s a bug, sometimes it happens by chance and the PO decides to keep it because it forces users to be tracked.

I completely agree with this post and I also feel burned out by ~~assholes~~ making economical decisions to build software. I've decided to fight it by writing.

My motivation was: Can I flip the engineering mind towards understanding that well-built software can be a reasonable economic bet too?

My article is called "On Technical Yield (and Technical Debt)": [link redacted]

I've also been annoyed about scope creep as it's been my manager colleagues favorite micromanagement argument. Scope creep can be a great thing: [link redacted]

Let's just do things differently.

Longtime lurker, I just created an account to respond to this thread.

Why not consider how much software has improved our life collectively and only find faults in it.

Developer is someone who knows how to instruct machines. They are not the owner of the product. So it might not their decision to disable right click.

The era that author talks about when everything was achievable with "html form", we should also consider the number of people who had access to the internet at that time. Also most of the people were technologist or early adopters. The usecases have changed in last "28 years" that "HTML forms" are not enough. The problem they solved in 1993 was good for that time but it does not always fit for the current standards and requirements.

There are 1.7 billion websites[1]. If we apply "Six Sigma" to it, still there would be 5780 websites would be defected. what are the chances that author might be browsing only those sites /s

I'd like to close with this - we have made huge advancements as a humankind and many of them seems impossible imagine without software developers. I'm grateful for all the technologist before me for their contributions even if its just "JAVASCRIPT-POWERED CUSTOM TEXTBOX WIDGET".

That is a consequence of corporations taking over "open" source projects. They hire too many people, many of them clueless. Because of "equity" no criticism is allowed and all "contributions" are merged.

We have a gigantic unstable churnfest with the most ridiculous unreliable half baked features getting you a promotion. Bonus points if you "grow the team"!

  > Grab a notepad and make a note every time you encounter some software bug in production (be it yours or someone else’s), or need to rely on your knowledge as a computer expert to get a non-expert system to work. Email me your list in a week.

Sorry, writing that list would take longer than I spend actually using software. Evolution telling me I'm "offline" because the DNS resolution isn't working because of some resolv.conf DNS extension gobbledygook. Community members seriously suggesting to blow away configuration which took a long time to establish in order to maybe fix some completely broken desktop application. CLI tools doing their own hard line splitting because they don't understand how terminals work. Web sites broken in any browser other than Chrome (I could have sworn the IE6 days were over). I can find about 10x more bugs than I have energy to even report, considering how completely broken most bug reporting processes are.

I think different categories of software require different mentality. It's ok for a user-facing innovative software to be built in the "move fast and break things" way. On the other hand, it is totally inappropriate for a filesystem or almost any firmware.

You need to strike the right balance for your use case between reliability and being able to deliver new things.

Windows 10 Settings menu has no error checking. If you disable "critical" service like StateRepository you wont be able to start Settings. It will just crash!

    Faulting application name: SystemSettings.exe, version: 10.0.19041.789, time stamp: 0x4aa1ce82
    Faulting module name: KERNELBASE.dll, version: 10.0.19041.906, time stamp: 0x2f2f77bf
    Exception code: 0xc000027b
    Fault offset: 0x000000000010b2dc

If you disable something more subtle you can even get Settings to crash with stack buffer overflow :o and it wont be logged as an Error! Just a friendly Application Popup information Event in the System log:

    Application popup: SystemSettings.exe - System Error : The system detected an overrun of a stack-based buffer in this application. This overrun could potentially allow a malicious user to gain control of this application.

> Every morning, I boot, then immediately hard-reboot, my workstation, because it seems to jigger my monitors into waking up properly to do their job.

I know Drew reads thread here (and I already sent him emails in the past) but... I've had similar issues over the years and typically these can be solved using several tricks: for example not using NVidia GPUs (and hence not using proprietary drivers).

Or configuring the system once to boot into text mode, then running "startx" from text mode.

Or booting into graphical mode, but then using a shortcut to switch to console mode, then back to graphical mode.

Similar shenanigans may happen too with the various types of sleep/powersaving modes. But I think they're all solvable or, at least, there are always workaround (not that Drew's "boot twice" workaround is that bad seen how fast Linux boots nowadays).

Or just not using Linux. He seems to be generalizing from "my setup has unreliable power management" to "software is unreliable". No. Linux PM is unreliable. Macs never fail to wake from sleep.

I've never read the source code of a database system. I don't think that will ever pay off. I have learned Postgres and SQL server pretty well. I'll pick either if given the choice. But reading the source would not help me.

There are a few valid points, but also unfounded pessimism.

The amount of stable libraries and stable innovative products is amazing. It's normal there's chaos, but what is good will survive the test of time.

Probably the result of companies like Facebook being mainstream popular who propagate mantras like "move fast and break things". I'm left thinking if there are other companies with Facebook-level success who preach reliability... All I can think of is Apple but I'm not even sure if they're already the best example. Assuming they are, it seems either "moving fast and breaking things" and making reliable products can make economic sense, but why do people tend towards the Facebook model?

I think it's important that the users have a choice: Do I want more features, or do I want more reliability? (Here I'm especially worried about ecosystems such as Apple that already limit user choice much more than this.)

After that, it comes down to whether businesses are customer-centric or profit-centric and why those are not the same. (I'd say the willful exploitation of people's personal data is much worse than any lack of reliability.)

Fully agree with the author except two things.

First, I think he is biased by his own job and missed looking beyond the curtain: the problem he describes is not specific to software development. What he describes as "Institutional recklessness and callous lack of empathy for users" is, in my opinion, much more widespread than just software development. I observe this almost everywhere: my health services, my food, my furniture, my electronics, my transportation, my holidays, etc.

The only industries where I still observe some degree of reliability or excellence are products/services heavily regulated and subject to the most stringent QA processes, and sport athletes. Anything else that is not thoroughly tested, vetted, certified and constantly subject to systematic rejection due to failure to achieve a high standard ends up being shipped broken.

Second, I do not believe a call to arms is necessary. At all. The reason for this, in my opinion, is because most people seem to be fine with how things are. I guess some wish they could get reliability, but I bet most will never fight for it. Many people do not even realize that they deserve to be treated better as customers/consumers than they are, in my opinion.

So, yes, while I agree with the general thesis (software is shipped broken, devs do not care anymore, stuff simply ends up being fixed iteratively through constant alpha testing by consumers who paid for a reliable product), I disagree that there is anything to do but surrender and start playing by its rules.

I have been working as a technology consultant for more than 20 years. I noticed this trend of increasing mediocrity and fought it for years. Only result is I am the one getting tired. Recently, I started caring much less about my quality of work. I stopped working late night to complete writing my reports or learn stuff, I stopped doing exhaustive research to give the best recommendations, I stopped giving presentations and sharing my knowledge with my peers, I stopped throwing new ideas or improvement suggestions at work, I stopped reviewing thoroughly any document when asked by a friend or colleague, etc. Guess what? I get constantly rewarded for being mediocre: more money, customers never been happier, colleagues feel more secure around me, I get asked much less questions by my boss and I get plenty of time to play video games.

Welcome to mediocre-istan.

Or maybe browser vendors can invest in high quality, easy to style, feature rich built in components, so we don't have to reimplement the wheel.

Its 2022 almost, and we still can't get a html select that has 20% of the features that chosen (https://harvesthq.github.io/chosen/) had 8 years ago.

Ironically I recently gave up using aerc, an email client that the author had a part it writing, because it kept crashing on my machine. I know its a WIP and there is a disclaimer warning of bugs but I thought it was a little funny. I do agree we should be doing better as a industry, but I'm not sure saying "hey everyone do better!" is going to have any effect.

This is one of the reasons why I like blockchain development: web / app developers are used to release shitty software and then play whack a mole with the bugs. That's a luxury that embedded systems developers dont have. Similarly, blockchain devs have learnt that once you publish your crap in the blockchain, theres no way to update the contract.

I guess developers should own the feature not just the implementation.

Unfortunately the impression is that people won’t use such systems

The irony of this been by a proponent of Wayland, which instead of fixing whatever issues were with X11 decided to nuke everything and rewrite it from scratch - such rewrites are some of by far the most common sources of bugs. And also one of many developers' favorite things to do since they replace something with known issues with something with unknown issues but of course they believe those issues wont exist or wont be as important (which of course can only be judged after the issues become known but hey, old stuff suck and new stuff are amazing).

> Prefer to use proven technologies.

LOL.

What he writes is not wrong, but i just can't get over that he is the one writing all that stuff when he is among the people who contribute to bit rot and buggy unstable software that comes from having to rewrite shit over and over just to do the same thing and keep up with the "~assholes~ developers" (as he calls them) who break existing systems (and i'm sure there will be some excuse about why in Wayland's case it wont matter and isn't the same thing).

Yes, just fix whatever issues were with X11. Why didn't anyone else think of that.

I for one can't wait to have Wayland+pipewire in full effect.

> Yes, just fix whatever issues were with X11. Why didn't anyone else think of that.

You need more than thoughts, you also need people to work on it and projects like Wayland that take the oxygen out of the room make that harder.

> I for one can't wait to have Wayland+pipewire in full effect.

And so people who need to work with the Linux desktop will have to implement support for X11 and Wayland and yet another audio, video and whatever API (and thus all the potential bugs you might introduce) or use some external library (thus introducing additional points of failure for the future and introduce external factors that can affect your project's goals) that tries to provide a layer over these (with its own potential bugs of course). In other words making things more complicated and increasing the bug surface.

I get where you are coming from, but honestly, have you really looked into pipewire at all?

It implements the APIs it is replacing. Apps do not have to be rebuilt, but they're getting an implementation upgrade. I sorta doubt the Wayland plan is to simply discard all the X11 apps.

Of course i have, PulseAudio also did the same with ALSA and yet that wasn't without a lot of issues for many people - and now Pipewire implements PulseAudio and ALSA and other stuff.

Similarly, yes XWayland does provide support for running X11 applications (though not for window managers, etc) but it also has issues of its own.

I mean, no? PA provides some interfaces through ALSA, but that's really not the same architecture change as re-implementing the whole thing but providing the same ABI. PA is also objectively terrible and that's one of the motivations for writing pipewire.

Seems to me like it's not rocket surgery to develop an X server that uses the Wayland compositor. It's just another video card at that point. We have way more complex things that work, so proposing that it won't work is not that interesting.

https://wayland.freedesktop.org/xserver.html

True. But there is another fact to consider, the target for products is: they have to be good enough to be sold. If 90% of people are happy with it, you won. Even if it’s far from perfect.

Everything on top of that is just sugarcoating.

> I know that when I middle click a link and “javascript:;” opens in a new tab, an asshole a developer wants me to left click it instead.

It frustrates me when developers actively break things like this

Yeah nobody cares about building fast reliable software. That must be why Rust and Ada are so unpopular at the moment.

It ain't the devs, its the managers and execs.

Our whole management class in this country is broken.

> assholes

> stupid

OK that's enough I'm not reading some frustrated ramblings.

Not developers, but rather their managers.

The reason I promote Radical Simplicity.

File under Tech Debt?

The post ultimately seems to boil down to "be more empathetic and considerate of your users", and building reliable software is of course one of the ways to accomplish that.

Unfortunately, the OP's points stem from various biases, and the idealism is harder to accomplish in practice than the OP makes it out to be.

> Almost all software that users encounter in $CURRENTYEAR is straight-up broken, and often badly.

Always been the case, whether now or decades ago. Many issues have been resolved; however, new ones will inevitably continue to appear. This is because the complexity of software will always increase, primarily as a result of market demands.

> My web browser has been perfectly competent at submitting HTML forms for the past 28 years, but for some stupid reason some ~~asshole~~ developer decided to reimplement all of the form semantics in JavaScript

Repeatedly unprofessional tone aside, HTML forms in their simplest form come still come with their own limitations and quirks.

Case in point, I've recently had to build a rudimentary form for uploading multiple files. By default, a file input field requires the user to select their desired files in one go, either by using Ctrl/Command or by selecting the files.

But, how do you allow users to drag and drop files from their file manager into the form? How do you allow users to select some files in one folder, then other files in another folder? What about an even simpler scenario: how do you let the user click on the "Browse..." button to select files, without overriding the previous selection? JavaScript is the answer to this.

Luckily, this specific problem doesn't require drastic solutions, though it hopefully does bring light to the fact that HTML forms aren't as perfect as they could be.

> Finally, stop letting economics decide everything you do. Yes, developers have finite time, and that time costs. Yes, users with annoying needs like accessibility and internationalization are more expensive to support than the returns they produce. You need to pay for it anyway. It’s the right thing to do. We can be profitable and empathetic.

Although I agree with the principle, the practical circumstances make it difficult to make this a reality. Many developers aren't full-time OSS developers like the OP is; instead, they work under an employer with demands, deadlines and restrictions in place.

This includes having to implement aggravating things like ads and abysmally re-invented forms, or having to sacrifice empathetic features for profitable features. In an ideal world, this shouldn't even be a problem, or at the very least a developer should speak up or change jobs when users' rights are compromised. Unfortunately, locations and opportunities can make this difficult, never mind having to keep the family fed.

There are of course many scenarios where it's the developers making incompetent decisions and being architect astronauts. But the problems the OP mentions are not caused by that; like they said, they are caused by economics. Economics shouldn't be the dictator of technical decisions, but unfortunately for most developers, it is.

This is why I believe that most of the user-hostility & brittle complexity is a result of a deeper problem, which is the developers not having enough liberty and resources. I'm confident that most developers didn't implement user-hostile features - including intrusive ads and awful forms - because they wanted to. It's easy to blame the developer for building the things, but in most practical cases, they built based on what they've been requested, not on what they've decided. I'm sure a developer will dedicate much more time and resources into robustness and friendliness if they were given the opportunity to.

Luckily, I have full control over all the technical decisions on the software I build for my employer, so I do strive for simplicity and robustness as much as I can, but I'm all too aware that not every developer has that luxury. I hope the OP will realise this, too. This isn't to spare developers from responsibility and accountability, but to bring to light that the problem the OP poses is much more nuanced, both in its causes and in its details.

The title misspelt “managers”

>Every morning, I boot, then immediately hard-reboot, my workstation, because it seems to jigger my monitors into waking up properly to do their job.

Maybe stop using Linux and wayland?

I've never seen any kind of computer system that can drive multiple monitors reliably. Sleep, disconnect, awaken, reconnect, etc. and some simple combination will end up with a monitor that sleeps, or the resolution is wrong, or other more amusing issues. It's 2021, and this doesn't seem like too hard a problem, but here we are.

Reply on news.ycombinator.com