Our software must get better | Hacker News Reader

Our software must get better

sethgodin.typepad.com

100 pointsjwblackwell59 comments

I dig Seth Godin a lot, but the real reason that software trends toward mediocrity over time is that the code base generally becomes an unmaintainable pile of crap. I find this this takes place over the course of years due to:

1. Tight development timelines that encourage features getting slapped in quickly at the expense of proper refactoring or architecture.

2. A shifting or unclear product spec - sometimes so dramatic that refactoring or rewriting is prohibitive (a hack is born)

3. The fact that many developers will touch the code over time. Not only will developers have differing code styles, the preferred code styles and patterns of the language and framework change over time too. The attitudes and level of commitment changes over time as original developers leave and new ones come in. If the code finally gets shipped to an outsourcing firm, you can bet that any last traces of quality are going to disappear.

This doesn't just happen at lazy or cheap firms - I've seen projects with an incredibly strong development team devolve over time too. It can be prevented, but it happens more often than not.

PayPal? Imagine the twisted bunch of cgi they probably still have lying around in their core code base. I'd be willing to bet there are still traces of Peter Thiel and Elon Musk in there, and nobody wants to touch it

The problem of software quality is complicated and difficult to reduce to a single set of issues. I don't think anything you've said here contradicts what Seth wrote.

I think he's right, though, that economics is the fundamental problem. We bang out code quickly to meet deadlines measured in months despite the decades-long impact a poor decision can have (see JavaScript for example). When we find ourselves slowing down, we add more people to write more code; why are we surprised when our software becomes bloated?

But if we missed our deadlines too much, we'd lose the market to our competitor, who wrote their software more quickly. And if we didn't hire more programmers, we'd lose again to our competitor, who hired tons of people to add attractive features.

It seems impossible to write good software without losing in the free market. IMO we should be thinking about how to support software projects outside of market-based corporations.

JavaScript seems an odd choice for an example. Sure, you can look only at the oddities of the language, but you could also look at it as the choice to allow dynamic scripting in web pages. At that level, it has been a hugely enabling choice that I would not label as a poor decision.

That is, there is no real evidence that any other choice would have obviously gone better. Why assert it?

Objecting to his choice seems odd. There is plenty of real evidence that Javascript has many conceptual and implementation issues and is, generally, poorly thought out. That it has redeeming features is not in question, nor an excuse.

Why object to it?

A few days by one person was spent on writing v1. Hundreds of years have been lost dealing with poor choices made over those days. That's not to say they did a poor job, just the timeline was crazy.

My point is you are focused on the losses, with no guarantee that other choices would have had the same gains.

Basically, this is an assertion without a valid comparison to back it up. I've become too much of an empiricist in my age.

At some point it would have been to late or to complex etc. But, an extra week would have more than doubled the time he had to work with. Pretending that's going to give anything close to an optimal solution is silly.

See my https://news.ycombinator.com/item?id=11487318 on software mutualism.

Three other social aspects I can think of that contribute to bad software are:

* "We have to do something!" - The idea that software can't be done and need only minor modifications/updates in order to be relevant and useful. Churn is confused with progress (see parts of the javascript community, or desktop software after 2000)

* "This code sucks, toss it." - Something that I'm certainly guilt of, in violation of the Chesterton dictum “Don’t ever take a fence down until you know the reason why it was put up”. This is in tension with the third point you mention.

* "Polishing sucks." - Once something is done, polishing (refactoring, simplifying and tidying) can be viewed as tedious and unbecoming of a creative genius such as ourselves. Large amounts of mental effort are put into small simplifications. Of course, over time, these simplifications compound and lead to truly special code, but nobody really cares about that. We are too busy learning the next framework.

I think the first one is 95% of the explanation for the state of iTunes. iTunes was essentially done a long time ago, but it's rare to encounter software teams that will go to their manager and say "our product is perfect. we shouldn't work on it anymore".

And if you have a product that's done but you still need to work on it, what do you change? You change the parts where improvement is most subjective and hard to measure: like the UI.

It is also rare to find managers that will stop monkeying with the product.

I think another big, big factor you're missing is people wanting to justify their existence, or "change for the sake of change".

For proprietary software, we see this in several ways: existing teams and managers want to keep their jobs, so they're not going to say, "this product is excellent and we're calling it done. We recommend that you keep Joe around to maintain it, and lay off the rest of us because we're not needed here any more. Joe will report to a different manager, so you can lay off Jim too. Sorry Jim, good luck finding another cushy middle-management job elsewhere!"

The other, related dynamic with proprietary software is wanting their software to seem "up-to-date", and giving customers a reason to keep "upgrading". Is this year's Photoshop really that much better than the one from 2 years ago? Probably not, but you're not going to get customers to keep paying over and over for your product if you don't keep making "upgrades". So you end up shoving all kinds of bloat in there and then trying to sell customers on paying for an "upgrade".

You don't see a lot of this stuff as much in the F/OSS world, since for a lot of projects, people aren't getting a salary for their work. But F/OSS isn't immune to it: take a look at Gnome3 for a prime example of fixing something that wasn't broken (and ending up with something that's far worse). But the causes for this aren't quite so clear-cut as in the commercial world. Part could be due to idealistic developers wanting to prove they have a better vision. But part could also be that, combined with getting a salary for it: most of the prominent Gnome developers are employed by Red Hat, so they've managed to sucker someone with money into funding their pet project. You don't see this with a lot of other F/OSS where there is no real funding. vim, for instance, has been chugging along for a couple decades now with not too many changes since it's mostly been done by one guy. There are a couple projects to replace it, but they don't have big corporate funding so they're just products of small teams of idealists and aren't moving that fast. A lot of F/OSS projects just don't have much work on them at all; they're in security maintenance mode only, and only see updates when someone finds some little bug or a security vulnerability. When was the last time TeX had any noticeable changes? Yet lots of academics use it every day.

With both proprietary and FOSS code, the architecture astronauts gets all the praise, while the code janitors go ignored.

End result is that doing maintenance gets ignored, while features and bottom up rewrites happen at irregular intervals.

Perhaps the one thing that has made the Linux kernel so popular, while user space seems to get reinvented over and over, is that Torvalds insist that once a user space interface has been established, it stays in, unchanged, as long as someone somewhere use it.

But user space itself do not hold themselves to that policy. And thus you have all the fretting about the program devs wanting the latest libs because older versions have a different interface etc, even between minor versions.

Damn it, look at Microsoft. Only with Windows 8 did they retire the 16 bit Windows stuff that has been around since Windows 3.1 (or even longer), and even then only on 64-bit installs.

This is why MS keeps being used around the world. Not because the UI is nice and shiny, not because it all those user helping niceties, but because people can continue doing their thing their way.

What are some examples of well maintaned/upgraded software, if any, that you know of?

I think the problem stems from the customer not caring about the quality of software. That combined with wanting it for nothing. They may say they care about it, but when you look at their behaviour, and not at what they say, they really don't care at all. It could literally delete all their files and most would shrug and go, oh well. What are you going to do?

When software users really start to care, and are willing to put their money where their mouth is, then you will see general software quality improve.

Also, people who pay for the software are not the same ones who use it / care about the quality. This may mean different things too:

- when outlook sucks, you're likely not the one who bought it for the company and they in turn likely don't even have a choice - it's a requirement

- when your company accounting package sucks, again you're unlikely to be the one who decided to buy it - it's a company / budget / b2b sales decision

- when the printer driver is bloated and sucks for 10k reasons, you're not the one who paid for it - you bought the printer, the company producing the printer bought the driver (paid for creation to requirements) and you don't get a choice

- etc. etc.

For a single person it's hard to bargain using money. I won't pay for Windows. My laptop came with it anyway and part of the price went back to MS. Even if I cared about the quality of Windows, there's nothing I can do at this point to influence it.

The big software packages are just completely detached from their users at this point. As long as they match requirements and don't break on global scale, they're not going to change.

You could influence Windows by buying a Mac instead. And arguably that has happened. Before OS X started taking off, Windows Longhorn was about trying to do object oriented filesystems and other stuff. It ended up as Vista, with semi-transparent window borders.

Really interesting perspective. In this example, external competition created a worse product than if they were left as a stagnating monopoly?

Before OS X the Windows team were hardly focused on the UI at all. OO filesystems weren't going anywhere. At least Vista shipped and had some new features.

> OO filesystems weren't going anywhere.

Really? Have we forgotten Smalltalk machines so quickly?

Smalltalk machines?

The question is not did we forget, but who had anything to remember about them in the first place?

Time for a car comparison.

People frequently will pay extra for high quality (or at least the perception of it). The problem is that in the presence of network effects, they usually won't.

Cars are a great example here: they're expensive, they're complex, and there's a huge number of different models available. If you just want the cheapest thing available, you can go buy a Chevy Spark or whatever the current bargain-basement model is. But people don't, they'll spend more money on a Japanese car because they think (rightly so, IMO, and also based on reliability records) it's higher quality and more reliable than an American-brand car, even though it costs more to get one with similar features. Or they'll buy a German car for similar reasons.

But when you buy a Honda or BMW instead of a Chevy, this isn't going to hurt you in any way: you'll be able to drive on all the same streets, park in the same parking spaces, etc. You might have a more limited selection of mechanics.

Same goes for, say, clothing or shoes. You can buy some cheap crap from Walmart, or spend a lot more for something else. But you're not going to be prevented from walking into a certain restaurant because you bought a $100 pair of shoes instead of a crappy $15 pair of shoes.

This isn't the case with software. If you decide Windows is horrible junk (esp. with Windows 10 and its spyware) and decide not to use it, you're suddenly going to be prevented from using a lot of software.

I don't think people understand quality of software. They'd definitely care if it deleted their files - I have seen this happen - but there's really no way to assess this in advance, no recourse if it happens to you, and insufficient choice to enable you to refuse to use something ever again if it has failed you. If e.g. Adobe Creative Cloud is terrible (which it is), but required by the industry and your collaborators, what choice do you have?

We're back to Akerlof's Market for Lemons economics here.

This is what happens when markets are run as oligopolies.

Software is bad because software has been run as a cowboy business at all levels. Power in the market is so asymmetrical that only the justice system has any hope of bringing it back into balance.

A change in consumer law to put oligopolies on the hook for negligence and consequential damages would magically improve software quality in short order.

I think it's a bit more than that, but overall I agree with what you're saying. Package managers, config managers and popular distros have been getting to me more and more lately because of their lack of understanding of polish or acceptable standards. It just seems like software developers, free or not free, are trying to push out as much as possible without caring about their base users.

Examples:

aptitude has maybe 8 different binaries to do essentially the same thing. Hell, for whatever reason, doing a search with it and piping to grep throws a message to stderr saying that grep isn't fully supported yet. Wtf? It's 2016!

pacman has a separate binary for searching called pacsearch. My natural instinct is to do a "man pacsearch", but it doesn't exist. The only parameter that exists is -n for stripping color, because apparently this is an acceptable default when grepping: ESC[0mESC[1mESC[35mcore/ESC[0mESC[1mpacman ESC[32m5.0.1-2ESC[34m (base base-devel)ESC[36m [installed]ESC[0m

salt's man page has 134854 lines in it.

archlinux released a kernel compiled with a flag that completely breaks sound on one of arch's most commonly used laptops. That was 2 months ago and the community recommends recompiling the kernel. wtf. I only just noticed a couple days ago, since I wasn't able to do a full upgrade since pacman will fail if you install python packages with it - which get installed from pacman packages. Ugh..

gentoo's documentation is ancient and has typos, grammar issues and places where simple Y/N questions are difficult to figure out. I really wanted to give them a chance....

ubuntu is.... hahahahaha. hah.... hah.

systemd and anything associated with systemd is so embarrassingly difficult to use. Why does an error message that tells me to run journald commands have to be longer than the actual error?! What happened to just printing to stderr :(..

The flaky user experience of UNIX software is not a new observation:

https://www.jwz.org/doc/cadt.html

Wow, cynical much?

I've been running linux professionally as a sysadmin for going on 10 years now and have had more consistent and mindfuckingly time wasting issues in the past year than I've had in the past five years.

"That wound is really starting to get bad, man. It's almost looking gangrenous" "WTF MAN THIS ISN'T A NEW OBSERVATION UGH"

If it was not so notoriously short on staff, and thus lagging the user space "state of the art", i would point you towards Gobolinux.

They care about the extrinsic quality of the software. A lot. Usually in a very narrow sense of "did it help me do what I wanted to do?"

Now, the intrinsic quality is an interesting beast that is much too subjective to be of any use debating. As soon as you find yourself debating the intrinsic quality, it is a good time to question if you are really trying to serve your customers.

1. one piece of software can be used by a billion people, no extra cost per person. Unlike candy or anything physical, it doesn’t cost more per user (not a penny more) to have more people use great software instead of settling for good software.

I strongly believe that you could spend all of the money on Earth on a software project for a billion users and you still wouldn't have something that all of the billion actually like and enjoy using. There's so much cultural variation between people globally that there simply isn't a pattern that fits everyone.

If you want your software to be universally accepted as something good you have to accept that you'll need lots of different versions of it.

To that end, Seth's assertion that "it doesn’t cost more per user" is quite wrong.

Yes, it's argument from the classic "80/20 myth" post by Joel Spolsky (http://www.joelonsoftware.com/articles/fog0000000020.html). The more people you want to reach with your software, the more bloated it has to become to be successful. The only choices are to accept and embrace that fact, or split the feature set and write variations on the software aimed at different niches.

I think it's possible to write opinionated software; often it turns out that those must-have features aren't actually must-have at all. Joel's own Trello is a good example of something that severely limits its functionality compared to its competitors, and ends up much better as a result.

It works because he is targeting a subset of all users. He is willing to give up on some potential customers who want some features that the product does not offer. He is also willing to lose actual customers who outgrow the product to competition.

I think it works because he hasn't yet reached the critical mass of users when it doesn't. It is probably a good strategy ("do things that don't scale").

GitHub did the same thing, by the way. Remember slide 56 of this slide deck from 2011: https://speakerdeck.com/holman/how-github-uses-github-to-bui...? Since then a lot happened, including https://github.com/dear-github/dear-github.

Regarding another of Joel's products, this was the top comment on the HN thread "Why Fogbugz lost to Jira": https://news.ycombinator.com/item?id=10314403

I think his argument is more that there are pieces of software out there used by huge numbers of people, and that this means improvements to them can benefit huge numbers of people. These improvements can be things like fixing bugs -- improvements that can be beneficial to users independent of those users' particular tastes.

He keeps using the word "better", but then the prime example he cites is "Roon" vis-a-vis iTunes. Go to their site[1], and check the comparison of their superior user experience (...) with a "meh" counter-example.

And you know what? I actually like the "meh" a lot better. iTunes used to look that way in the beginning, before they went all color-crazy and web-like.

There are certain properties of software where you can clearly label a competitor or new version better: Fewer bugs, faster operation, no superfluous user interaction. But beyond that, we quickly enter the areas of taste, not quality. And, y'know, de gustibus...

[1]: https://roonlabs.com/

Plus I notice Roon operates on a subscription or one off 'lifetime' model for a year (or however long it takes for them to go bust'. No mention of what happens if you decide to stop using it. Is your library catalogue forever locked to their particular listing format if you want to move on? Probably.

> Fewer bugs, faster operation, no superfluous user interaction

The only music player which fulfills all of this and much more is foobar2000. There is nothing else which comes even close to it.

Looking at the Roon webpage:

> Forget everything you know about music players. Music is an experience, and Roon reconnects you with it.

Experience? Why does everything has do be an experience? I just want to manage my library and listen to music and not fight against miles and miles of whitespace and padding and "beautiful graphics" to actually find my content.

> We have more convenience than ever, but no feeling of excitement or engagement.

I don't want to be excited or enganged, I just want to listen to music...

> What you get is a searchable, surfable magazine about your music.

Searchable? I bet that foobar2000 is way more searchable than Roon will ever be.

> “It’s light years beyond any current app.”

lol

Perhaps I am just in the wrong target demography but there is not a single thing on that page which would convince me to switch to Roon. foobar2000 is the vim/emacs of the music player world and everything else is just a copy of notepad.exe with some fancy graphics.

I've been quietly advocating the mutual model for a while now.

Free (Libre) Software is great where it works, but that seems to be areas where the users are also programmers. You can't really gain the benefits of software freedom if you're not a programmer, so your experience degenerates into using it because it's zero-cost and your best chance of getting a change you want is to complain on a hostile mailing list.

People say that non-developer users should be able to hire developers to sort out problems in Free or Open software, but in practice this very rarely happens.

In a software mutual, the source would be closed but with a guarantee of opening it up if the mutual could not continue. Possibly with an option of source licenses for mutual members. However, there would be two key differences from a Free project:

- users who'd not paid wouldn't recieve the benefit of the software; no free-riders (modulo piracy, obviously)

- development choices would be made by vote of paying members, not just developers.

I think a software mutual aid organisation could be GPL-compatible, too: it'd be one legal corporation whose members (human or incorporated) would be using the software collectively.

It'd be an interesting idea, surely.

That's pretty interesting as a model.

Is voting the right way to influence developer choices though, or is a more market based solution the right approach (like using subscription fees and thus loss of fees = feedback that the decisions are wrong).

I was thinking of both, really: pay for software, get a vote in its future. Possibly split into an upfront + ongoing membership payment, as software needs long-term maintenance. People who haven't paid wouldn't get a vote.

The intent is to fill precisely those areas where the current software market has failed, so we know that a pure market solution is not going to deliver what a particular set of users want.

After reading a lot of Christopher Alexander I came to the conclusion that the key is to build for people.

This applies to buildings but also to software.

I think Seth is talking about this.

So don't build (sofware) for:

  - status
  - money
  - cars (urban planning)
  - the looks of it (using art is no fun)

Unless of course the people you're building for are actually interested in status, money, cars and aesthetics (ok I know you didn't mean their status and money but anyway).

Unless you need money to live on.

Also see a the discussion on HN: Why Shitty Products Survive (and Thrive!)

https://news.ycombinator.com/item?id=10994721

tl;dr: The software market is still largely free and unregulated, and as a result, the tradeoffs made in producing software tend to reflect the preferences and priorities of many millions of different people from many diverse backgrounds. And these people don't share the preferences and priorities of Seth Godin, and that's why software sucks.

After reading through the comments, there is one more thing to throw in the mix: software can only be as good as the tools you develop them with.

I'm building the same app for Android and iOS, and the latter is ways quicker. Even with last week's new Android Studio 2, it takes at least twice as long to build the changes and run a new update on Android. (Not to mention testing with multiple screens, emulators etc.)

Not surprisingly that shows in developer's salaries, and imho the quality of the products.

Seth has written some great stuff in the past but this was not his finest.

    > 1. one piece of software... no extra cost...

There is a cost. Additional bandwidth for downloading. Additional language translations. Additional operating costs.

    > 2. fixing software today fixes it for everyone, in the world, going forward ...

This is terribly fallacious. In most cases it's impossible to make software always work because of external dependencies (for example government policy of time e.g. DST).

    > Alas, software tends to be mediocre. There are a few reasons for this:

Compared to what? I think Seth is myopic here and is not looking at the big picture. Most software is absolutely incredible in what it does. Ask and show anyone over a certain age that is not technologically biased and they are amazed what software is capable of.

Even gimmicky iPhone Apps are pretty impressive compared to where were 40 years ago.

Compare this to cars 40 years ago.

The rest of Seth's points basically apply to almost everything else in the world (from politics to food).

I just feel like I have heard this so many times (e.g. why does software suck).

So, none of the reasons that software is mediocre is that developing, and maintaining, great software is actually hard?

Instead he claims it's all just due to market (lock-in, free) and management (more people) reasons.

That strikes me as being somewhat cynical, but perhaps I'm not reading it right.

Personally I think the bar is pretty high for software, since it's all mainstream now it's hard to get by with under-delivering in terms of features, which makes even "simple" things kind of complicated, which often necessitates at least a slightly larger team than before. Having a larger team I think makes quality harder to achieve due to the "committee-factor".

Of course building great software is hard. So is building great buildings, or designing great planes, or producing great, internationally sold cereal.

The commitee factor is a problem in anything provided internationally to millions of users, as well as in anything sold for hundreed thousands of dollars to single customers.

I don't think software is inherently more difficult than other products with comparable scope and scale. If anything, in software it's easier to develop and provide a product with big scope or big scale than in most other fields. In software you can often achieve quality and scale at the same time with a team smaller than the one of your local hardware shop.

But I agree with the author that in software, like in some other fields, a lot of the incentives are not aligned to create great software.

> The commitee factor is a problem in anything provided internationally to millions of users, as well as in anything sold for hundreed thousands of dollars to single customers.

As with anything in life, things are never black and white but filled with grays. For example, I'd much rather my banking software or the avionics of a plane that I am about to board be designed and built by committee rather than the local hotshot cowboy coder who wants to rebuild everything from scratch with whatever happens to be the tech stack-of-the-month. Great software is often boring and un-sexy and not everything can be built by two developers in a bedroom.

> fixing software today fixes it for everyone, in the world, going forward (and for connected computers, going backward as well).

this would be true if reality didn't change underneath it. the people who are saying 'maintenance is where this breaks' are right.

Not sure I agree with the author that 'our software can get better' but it would definitely be nice. Software exists in an ecosystem and there are 'resource and competition' reasons for bad software that aren't visible if you just look at the codebase.

So can anyone else vouch for Endicia? We use Stamps.com for our shipping at work, and nobody really likes it. But I don't want to recommend something that is only marginally easier/faster/better. Is it truly a noticeably better piece of software like Seth says?

https://en.wikipedia.org/wiki/5_Whys

"But let's not pay any more for it, or pay devs any more."

I can not believe that he thinks software's biggest problem is that most of it is free. His description of users as "bait" whose habits will be tracked (which seldom occurs with free software, I can only think of fewer examples than the number of fingers on my one hand) seems to exclude the fact that paid software (hello! Windows 10?!) can do the same thing.

Anyone against free and/or open-source is an obvious shill or just plain greedy.

I don't think his argument finds problems with the fact that some software is free. Having a large user base means you have the ability and the responsibility to make an impact an a very large number of people. He speaks to both the user and developer that mediocre de facto applications with large user bases should not lead to complacency nor its acceptance.

I don't think he means free as in freedom.

Windows 10 is ""free"" (gratuit) to most users, and definitely part of the problem.

Don't call people "shill" without substantiation.

Reply on news.ycombinator.com