Hacker News Reader

Comment by CJefferson | Hacker News Reader

The problem is often the "bugfixes and performance improvements" are a list of 60 things, and for almost all users "there are less crashes and it is faster" is all they would want to know. How many users want to read (OK, possibly people here)

"On some versions of glibc, the 32-bit specialisation of memcpy which used SSE2 instructions had a bug when the memory being copied crossed the 2GB boundary, so in that case we now use our own memcpy"

Or "Changed some internal lists to be only be sorted when they need to be rather than eagerly sorted whenever elements are added, which speeds up start-up time".

View on HN

I think the key is describing the user-facing effects and impact. This means knowing your users (are they developers, IT or highly technical? Do they automatically or manually update quickly, or do they take time to evaluate before rolling out updates?). It's a way to convey continuing value to users and convince them it's worthwhile to keep paying, and get motivated to get the update.

I'd try to change what you wrote to something like:

"Fixed out-of-memory errors on some 32-bit systems"

"Improved startup time for configurations with lots of items"

This makes it easy to gauge if this update is important to get installed ASAP, such as fixing something they've experienced or are likely to experience.

Grouping "several UI fixes" or "performance improvements" is usually fine, but I tend to call out something like "fixed UI bug where a network error could result in changes not being saved with no warning" or an issue that a sizable chunk of users had reported.

Doing this well takes time, though, as you have to synthesize the internal ticket summaries, PRs and/or commit messages and reword almost everything. This is understandable for open source or free/indie apps, but for subscription/"enterprise" software it's definitely one of the differences between "great" and merely acceptable (or worse).

> but for subscription/"enterprise" software it's definitely one of the differences between "great" and merely acceptable (or worse).

I couldn’t agree more.

I tried writing more informative but easy to digest release notes when a couple noisy users mentioned at a trade show that they wanted to know more about our bug-fixing efforts.

To my surprise, we got a huge volume of communications from our users that they loved our new release note style and to keep it up.

It takes us about 1 hour / release to write. We release small changes weekly and large changes monthly to about 1mm MAUs.

> Doing this well takes time

For us, not that much time, as everything goes through the bug tracker (even enhancements) so it's mostly a copy-pasta job. When I submit a fix for a bug, I'll often edit the subject to accurately reflect what the problem actually turned out to be.

Even less time if devs add these to CHANGELOG.md with each user-facing change so that it’s already compiled for you.

For closed source systems detailed changelog is very nice. "Fixed USB reset issues" is close to useless, when you try the new firmware and it still exhibits USB reset issues. More detail the better. And describing the actual changes ("increased XYZ timeout on USB resume") instead of symptoms they tried to address ("we fixed the USB reset" - not) is actually much better.

> I'd try to change what you wrote to something like:

I'm used to seeing from well managed projects a paragraph saying that there's new feature A and bugfixes and performance improvements.

And then a detailed change log with big feature A, smaller features B & C, and 10 bug fixes described. Maybe 2-3 performance improvements that are expected to only -really- help with degenerate cases.

And then, there's a dozen things that aren't considered important enough to be in this list.

I think focusing on "user facing impact" is a good idea and something I'll be using in the future. One challenge is that developers don't always know the user facing impact of their improvements.

For example if I notice a logic error in a system and fix it, I may not know what if any software configuration could trigger the error. If it were obvious, QA would've found it. And given the choice between further investigation or leaving the bug unfixed, my employer would prefer the latter.

Or one time I was working on a new feature that required speeding up the system's implementation of malloc(). That improves the performance of the entire system to varying degrees. Pinpointing exactly where the user's experience improves would require extensive benchmarking outside the scope of the work.

There’s also a mitigating liability aspect to this. If you start providing minutiae detail, you may start revealing pretty big faults that have escaped to prod. People question the safety of your product and now you’re open to lawsuit or at least micromanaged by some unforgiving end-customers. To cope, you start to not fix certain problems so you won’t have to report the bugs or you start lying what the bugs were. I’m not so sure it’s on net a useful direction to be quite that open.

Aren't the flaws the sources of the liabilities?

Sure, but most people accept that flaws are inevitable. It's not practical (or profitable) for most industries to produce totally flaw-free software.

If you can't stop the flaws, the next best option is to shield yourself from those flaws harming your financial well being.

Having a changelog does not make you more open to lawsuits.

A detailed changelog wouldn't change the facts, but it just provides more free information to potential plaintiffs.

This explanation doesn't make me empathetic at all, and lousy changelogs are still lousy.

This is likely why useful and important details in changelogs are suppressed. Products where owners/managers value and emphasise transparency are much better at this, but it requires a specific culture. I mean - it does take guts to say "All media in the application could be downloaded without authentication by anybody on the internet. This has now been fixed."

Has there ever been a lawsuit in which a company was held liable for bugs despite the usual disclaimer?

Let’s stop condoning covering up incompetence.

If you fuck up, admit to it, fix it and learn from it.

If you fuck up bad enough that someone had a good case against you then you should make them whole and use the opportunity as a wake-up call to make sure this never happens.

Are there ever successful software product quality liability lawsuits?

Back in the 80s when I was starting out in the compiler business, I thought it would be a good service to include with it a list of the known bugs.

It didn't take long before a magazine reviewer did a compiler roundup and simply printed the bug list as his "review". It was a disaster for my business.

It took me 20 years to get over being brutalized by that and make the bug list publicly available again.

https://issues.dlang.org/

I had a competitor do something like this to me in the 10s.

We had a structured weekly/monthly small/big release cadence compared to their bi-annual update. When we started thoroughly documenting our bug-fixes publicly, they used that against us in negotiations on a large deal. Basically alluding that our software was buggy, seeing that we had something to fix every week. Luckily we thwarted that logical fallacy by asking how many bug fixes they did last year and had no real answer.

People suck. Glad to hear you’ve overcome the fear on this, it’s worth it and I also still maintain a public list of bug fixes.

The take-away from this is if a vendor doesn't provide a good changelog, it's fair to assume they're trying to cover up incompetence (or some other shortcoming).

With good automated testing, a bug recurring should be extremely rare. A piece of the software that constantly has bugs is a sign there's high technical debt that needs to be addressed. New features consistently not working could be a sign of a problem in the team/org itself (not the right people, enough people, or enough time allocated).

Even with all the best practices, big faults will make it through sometimes -- that's just the nature of most software dev. I think the best way to handle them is be transparent, but also be specific. "Fix bug where entire database can be corrupted" will cause you a lot of grief (rightfully so). Something like "Fix critical data corruption bug when saving a record containing specific trailing unicode characters on systems with libzip 1.3.3 or earlier installed" is much better and helps reinforce that:

1. You have thoroughly investigated the problem

2. The scope of problem is limited, and doesn't affect all customers (even though it could be most)

3. It's understandable how such a bad bug could happen and why testing didn't catch it

I think the author of the article brings up a good point regarding this, though. If you're a non-technical user, you're not going to read it at all. You might update, you might not, but you're not going to look through the list of changes.

If you're a technical user, you know it's unlikely that update == "more crashes and slower load times" so saying the opposite isn't helpful, and a list of things might be helpful.

So as a user of an app you probably fall into one of 3(ish) categories:

1. You read the changelog and something is relevant to you.

2. You read the changelog and nothing is relevant to you.

3. You don't know what the changelog is.

Listing the bugs that are fixed helps some of these groups. "Bugfixes and performance improvements" is a given, and helps no one.

Even when I'm in group 2 and don't want to read every update, I'm reassured by the fact that users in group 1 can find relevant information. I also sometimes move from group 2 to group 1 when I need to track down what version introduced something that's just now causing me problems. :)

> If you're a technical user, you know it's unlikely that update == "more crashes and slower load times" so saying the opposite isn't helpful, and a list of things might be helpful.

It's precisely because an update might bring more crashes that I'd like to see what changed.

> If you're a technical user, you know it's unlikely that update == "more crashes and slower load times" so saying the opposite isn't helpful, and a list of things might be helpful.

Umm, no; if you're a technical user, you know it is likely that update == "more crashes, slower load times and useful features breaking or disappearing entirely", regardless of what the changelog says, because you've observed exactly that happening with eg your webbrowser or image editor.

> "Bugfixes and performance improvements" [(]is a given[)], and [(]helps no one[)].

Well, you're half right.

I tend to fall in either 1 or 2 depending on whether I've been bitten by a specific bug and need to know if it has been fixed.

The worst changelogs are the ones that say "Performance improvements and bugfixes" only to find out the entire user interface has changed after updating.

Some non-technical users do read change logs, looking for new features. Even if they take every update, the notification of an update pings the hope that there may be something useful or fun. "Bug fixes and performance improvements" means "you need this update but there's nothing you really care about".

So there are least two kinds of updates, those with user-facing features and those without. Currently both come in the same stream. We could come up with a better way of differentiating the two, including not notifying users at all when the boilerplate "fixes and performance improvements" comes on. Just update the app and what the user doesn't know (hopefully) won't hurt 'em.

See Reaper (the audio thing) for the opposite extreme. The changelog lists everything, so sometimes you miss significant new features because they don't make a big deal about it.

https://www.reaper.fm/download.php#whatsnew

They put a whole notation editor in and didn't say anything past the item in the changelog. Most tools like it have a notation editor, but it's a $300 addon instead of a free update to the standard $60 package.

Doesn't have to be coverage of every minute detail down to the implementation, but a rough outline of at least the most significant changes seems like a good middle ground. On several occasions I've revisited a feature (or even an entire application) upon seeing that an issue I've had has been addressed in the changelog.

A lot of people here would like this level of detail, actually. But general apps are intended for the general population, which is why big orgs go for the minimum watered down version. It's more work and overhead if they confuse their users.

It's important to list what bugs you fixed. You can do fancy things like group your changelogs under headings if you want to make it easier to maintain focus for different audiences. A CAD tool I use, for example, lists every single bug fixed and references a ticket number with each. This is extreme, but much better than the opposite extreme.

It lets me know when I can stop using workarounds, or alert me to landmines I've unknowingly tripped and lets me go back to verify old designs.

Regardless of who the user is, a changelog is supposed to tell you what actually changed. If you're not going to do that, don't bother having one. But if you don't bother writing one, I'm not going to bother updating to the new version. From my (the user's) perspective there's no incentive to do so unless I know what I'm getting. There could be anything in there, so I'll stick to the devil I know.

I prefer the detail. You could always make the first line: “bug fixes and performance improvements” with details following. Then people who don’t care can ignore the details, but people who do care can get an understanding of what the change is about.

Deep understanding of a change is really helpful when looking through history or trying to make sense of why this particular line was changed, etc.

This is about app store changelogs, not commit messages

I have never understood "it's an app store" precludes an expander or link to details.

Two reasons:

+ Customers will often have no understanding of what you're saying, but still click the link and give low reviews for it's content.

+ More stuff the the app store review team to refuse publishing an update for.

So it is more important than ever because there are no commit messages you could go through, right?

Hehe I'd be happy to read such a changelog

I doubt you would. A large app each would have hundreds of changes like this. It would get pretty boring after the first dozen or so.

Look at SBCL's changelogs, I like those. I definitely would like that over "there are less bugs and it is faster".

Reply on news.ycombinator.com