Lets talk about changelogs (how I loathe 'bugfixes and performance improvements)
raymii.org
raymii.org
"On some versions of glibc, the 32-bit specialisation of memcpy which used SSE2 instructions had a bug when the memory being copied crossed the 2GB boundary, so in that case we now use our own memcpy"
Or "Changed some internal lists to be only be sorted when they need to be rather than eagerly sorted whenever elements are added, which speeds up start-up time".
I'd try to change what you wrote to something like:
"Fixed out-of-memory errors on some 32-bit systems"
"Improved startup time for configurations with lots of items"
This makes it easy to gauge if this update is important to get installed ASAP, such as fixing something they've experienced or are likely to experience.
Grouping "several UI fixes" or "performance improvements" is usually fine, but I tend to call out something like "fixed UI bug where a network error could result in changes not being saved with no warning" or an issue that a sizable chunk of users had reported.
Doing this well takes time, though, as you have to synthesize the internal ticket summaries, PRs and/or commit messages and reword almost everything. This is understandable for open source or free/indie apps, but for subscription/"enterprise" software it's definitely one of the differences between "great" and merely acceptable (or worse).
I couldn’t agree more.
I tried writing more informative but easy to digest release notes when a couple noisy users mentioned at a trade show that they wanted to know more about our bug-fixing efforts.
To my surprise, we got a huge volume of communications from our users that they loved our new release note style and to keep it up.
It takes us about 1 hour / release to write. We release small changes weekly and large changes monthly to about 1mm MAUs.
I'm used to seeing from well managed projects a paragraph saying that there's new feature A and bugfixes and performance improvements.
And then a detailed change log with big feature A, smaller features B & C, and 10 bug fixes described. Maybe 2-3 performance improvements that are expected to only -really- help with degenerate cases.
And then, there's a dozen things that aren't considered important enough to be in this list.
For example if I notice a logic error in a system and fix it, I may not know what if any software configuration could trigger the error. If it were obvious, QA would've found it. And given the choice between further investigation or leaving the bug unfixed, my employer would prefer the latter.
Or one time I was working on a new feature that required speeding up the system's implementation of malloc(). That improves the performance of the entire system to varying degrees. Pinpointing exactly where the user's experience improves would require extensive benchmarking outside the scope of the work.
For us, not that much time, as everything goes through the bug tracker (even enhancements) so it's mostly a copy-pasta job. When I submit a fix for a bug, I'll often edit the subject to accurately reflect what the problem actually turned out to be.
It didn't take long before a magazine reviewer did a compiler roundup and simply printed the bug list as his "review". It was a disaster for my business.
It took me 20 years to get over being brutalized by that and make the bug list publicly available again.
We had a structured weekly/monthly small/big release cadence compared to their bi-annual update. When we started thoroughly documenting our bug-fixes publicly, they used that against us in negotiations on a large deal. Basically alluding that our software was buggy, seeing that we had something to fix every week. Luckily we thwarted that logical fallacy by asking how many bug fixes they did last year and had no real answer.
People suck. Glad to hear you’ve overcome the fear on this, it’s worth it and I also still maintain a public list of bug fixes.
With good automated testing, a bug recurring should be extremely rare. A piece of the software that constantly has bugs is a sign there's high technical debt that needs to be addressed. New features consistently not working could be a sign of a problem in the team/org itself (not the right people, enough people, or enough time allocated).
Even with all the best practices, big faults will make it through sometimes -- that's just the nature of most software dev. I think the best way to handle them is be transparent, but also be specific. "Fix bug where entire database can be corrupted" will cause you a lot of grief (rightfully so). Something like "Fix critical data corruption bug when saving a record containing specific trailing unicode characters on systems with libzip 1.3.3 or earlier installed" is much better and helps reinforce that:
1. You have thoroughly investigated the problem
2. The scope of problem is limited, and doesn't affect all customers (even though it could be most)
3. It's understandable how such a bad bug could happen and why testing didn't catch it
If you fuck up, admit to it, fix it and learn from it.
If you fuck up bad enough that someone had a good case against you then you should make them whole and use the opportunity as a wake-up call to make sure this never happens.
If you can't stop the flaws, the next best option is to shield yourself from those flaws harming your financial well being.
If you're a technical user, you know it's unlikely that update == "more crashes and slower load times" so saying the opposite isn't helpful, and a list of things might be helpful.
So as a user of an app you probably fall into one of 3(ish) categories:
1. You read the changelog and something is relevant to you.
2. You read the changelog and nothing is relevant to you.
3. You don't know what the changelog is.
Listing the bugs that are fixed helps some of these groups. "Bugfixes and performance improvements" is a given, and helps no one.
The worst changelogs are the ones that say "Performance improvements and bugfixes" only to find out the entire user interface has changed after updating.
So there are least two kinds of updates, those with user-facing features and those without. Currently both come in the same stream. We could come up with a better way of differentiating the two, including not notifying users at all when the boilerplate "fixes and performance improvements" comes on. Just update the app and what the user doesn't know (hopefully) won't hurt 'em.
It's precisely because an update might bring more crashes that I'd like to see what changed.
Umm, no; if you're a technical user, you know it is likely that update == "more crashes, slower load times and useful features breaking or disappearing entirely", regardless of what the changelog says, because you've observed exactly that happening with eg your webbrowser or image editor.
> "Bugfixes and performance improvements" [(]is a given[)], and [(]helps no one[)].
Well, you're half right.
https://www.reaper.fm/download.php#whatsnew
They put a whole notation editor in and didn't say anything past the item in the changelog. Most tools like it have a notation editor, but it's a $300 addon instead of a free update to the standard $60 package.
It lets me know when I can stop using workarounds, or alert me to landmines I've unknowingly tripped and lets me go back to verify old designs.
Regardless of who the user is, a changelog is supposed to tell you what actually changed. If you're not going to do that, don't bother having one. But if you don't bother writing one, I'm not going to bother updating to the new version. From my (the user's) perspective there's no incentive to do so unless I know what I'm getting. There could be anything in there, so I'll stick to the devil I know.
Deep understanding of a change is really helpful when looking through history or trying to make sense of why this particular line was changed, etc.
+ Customers will often have no understanding of what you're saying, but still click the link and give low reviews for it's content.
+ More stuff the the app store review team to refuse publishing an update for.
The article says at the end: "You see? Not that hard right?". It is that hard in bigger organisations.
The size of the organisations is not important, as is the size of the team working on the particular feature/app.
Apple is huge, for example, but eg. Logic Pro has great changelogs.
We also see huge apps (with large teams) do better than than small apps (with much smaller teams). It's more about bothering/caring than difficulty.
They say there if there is a will, there is a way, so in a sense I agree that these companies probably could do something about it if they really wanted to, but I also don't think it would be easy, and considering how few people actually look at release notes, I am not sure the investment is worth it.
This is a terrifying sentence to me, because what it implies but doesn't say outright, is that there's not a single person in the organization who actually fully knows what is being released. If there were, then they should already have this information!
Honestly, if you don't have a release manager with a 30,000 foot view of the operation, you probably have bigger organizational problems than "it's hard to know what to fill in on the app store's release notes".
This is how retrospectives on supply chain attacks start, basically.
It’s not the collection of the information that’s difficult, it’s delivering it to the user in an appropriate way when you use phased rollouts with feature flags.
The change log that the App Store presents to the user is static and identical for everybody. The changes that a user actually experiences are not. So you might have 0% of your users experiencing a certain feature when the release goes out, 10% a day later, 50% a day after that, and 100% at the end of the week. Or, if the metrics show a bad response, it might end up at maximum 10% seeing it, and then it dropping back down to 0% the next day.
Apple gives you a static text field to enter the change log information, with limited length. If you have ten different features to release, your users will all receive them at different times, and not all of them are guaranteed to make it out to everybody, how is a static text field supposed to deliver that information?
If you don't put anything about new features, don't be surprised when users don't use them (because why would they know) or complain what something "breaks" (or just changes on their workflow) because your changelog communicates nothing.
That’s a guaranteed way to get users complaining that they don’t have a new feature that you “promised” that they may never receive. It also isn’t appropriate for multivariate testing and just isn’t going to be understood by most users anyway.
There are way to do this in a decent way, but you have to do it within the application using your own logic. The change log provided by the platform is inadequate.
> If you don't put anything about new features, don't be surprised when users don't use them (because why would they know)
If you need to tell your users about new features in the App Store change log, then you’ve already failed because almost nobody sees it. Users know about new features because either a) it’s obvious when you use the application, or b) you tell them about the new features somewhere they will actually see it, at an appropriate time.
> or complain what something "breaks" (or just changes on their workflow) because your changelog communicates nothing.
A change log does not prevent this. You are attributing a lot of power to the platform change log that it simply does not have.
That doesn’t mean it’s impossible to produce more detailed notes for a big application, but it’s a pretty involved process basically requiring a “release note manager”, not something that happens by default.
Sometimes there is, but as siblings point out, it's difficult or not cost effective to distill this information into an app store changelog blurb.
More often (in my opinion), it's the terrifying situation you describe. Nobody know what the heck is going on at an overall level. Which can work surprisingly well, but often means efficiency (network, memory, cpu, disk space) is very hard to come by.
You can't ship a quick update with Apple, because review isn't quick. Users who upgraded soon after release will be stuck with something broken until the new version is available.
You can usually ship quickly with Google, but users are slow to upgrade, but may have caught your first one, and not get your second one for some time.
Although I'll try to resist going on too much of a pro-FOSS rant, the change audit process is much easier for open source projects since you can read not just the changelog entries, but also the corresponding code to verify the claims.
That helps with various things including developer trust - it helps you determine whether a project and authors use test coverage to reproduce bugs and prove their fixes, whether performance improvements are measurable, whether there are risky side-effects to look out for, and so on.
Increasingly I think that these kinds of 'software due diligence' should become part of recommended software engineering workflows, and that makes me wonder whether some of the repetitive aspects can be automated.
A genuinely user-focused, trustworthy app ecosystem would, I think, allow some of these 'change audit' processes to be visible to users so that they can make informed upgrade decisions, and to create a marketplace incentive for upgrades to be reliable.
Instinctively, developers reach for automation to solve this problem but this is an editorial problem for most software. Code changes very rarely map to notable changes for your intended audience. Even with rigid system like conventional commits, you can't entirely automate proper human-focused changelogs and release notes.
You're going to have to clean things up and editorialize if you really care for your software users to understand what notable changes are included in a given version release.
If you can't think of any, maybe you're releasing too often. If it's impossible to tell things apart because there are so many changes as another commenter pointed out, I think you have a way bigger problem.
That said "Bugfixes and performance improvements" is as useless as it gets ...
Before my company implemented IAP in a few of our apps we did the same thing, but decided the risk was too great. Getting rejected is one thing, but getting your account terminated is obviously a much bigger deal.
It also allows app developers to bucket users in groups and A/B test experiments on users to see what works best. It would not be uncommon for the larger players (Microsoft, Google, Facebook, Pinterest, Uber, Spotify, etc) to have 100s or 1000s of experiments running concurrently. It also decouples rolling out features from App Store releases.
Having said that, I also hate generic changelog messages and not defending the practice. If new features are to be launched with the build, write it in the changelog and add "(rolling out)" or "(coming soon)" next to it.
And I don't mind reading 'bugfixes and performance improvements' ... but only if that's really what the new version is about. I like changelogs, but even for me a 'we fixed an issue where if you enter a 1025 characters text into the title textbox and press back twice while rotating the phone the app closes' is probably too much, but if you added a new option to increase the text size please say it, I don't want to check all the settings screens each time each app updates, and more often than not someone tells you 'hey, aren't you using this option?' No, no I wasn't, because no one told me.
Are you a big company and you have automated updates? Why not have an automated checklist of changes too? I'm sure you have a 'roadmap list', so use it.
My changes are mostly technical when I use the "bug fixes and performance improvements" text, but I tend to write more for the app store reviewers to let them know what is going on. I know they read it.
In the end my audience of users are very non technical. I am authentic to them, that is the image that I sell my apps with, and I am good at it. So they just don't care that I've updated library X, or improved error handling somewhere, or I support Apples latest devices now. They do have faith that I am doing the right thing, and they write to me as if they know me, and I like that.
I'm very passionate about this idea for open source software, especially things like ruby gems or npm packages that I might use as a dependency, but for some reason I can't muster the indignation to care about this for apps. Maybe it's because I care primarily that the app works and the latest version is stable. I'm generally not going in and considering upgrades to apps strategically, and even if I did I have no ability to see the previous changelogs let alone install those versions. If for some reason I want to avoid an upgrade now I'm living on borrowed time and need to start thinking about a replacement or carefully curate my backups so as not to lose access. Sure in a perfect world it would be nice to have perfect changelogs, but it's so far down the list of things I want out my apps that I place zero value on it.
Sounds like the screen shot he took in September really was a bugfixes and performance release.
So it sounds like these apps (at least, WhatsApp) follow a wise policy - tell the users what has changed that matters and don't drown them in irrelevant information.
I had the same policy for my teams - tell users 4-5 new features/changes to workflow they are going to see and for g-d's sakes, do not drown that out by listing 400 irrelevant infrastructural changes we happen to be pushing out.
Well, I would love to. Unfortunately neither Play Store nor App Store allow you to do that... so "bug fixes and performance improvements" it is, 99% of the time.
Individual freedoms and privacy have no future in the direction technology and society are going.
Just like when I see “50% less sugar”, I immediately assume they replaced sugar with chemicals.
One reason might be that they are not automatically generated. I hand-edit each entry.
Sometimes, I copy and paste the changelog entry into the last commit of the release (but I make sure to have a short, succinct initial sentence, as that is what shows up in Git browser history).
The changelog may end up being a lot more "generalized" than the commit comments, though, as it's meant to be read by users.
I appreciate the kinds of changelogs that you usually get with developer tools, where they are like encyclopedias.
Thanks for your contribution to society, Apple Arcade.
For example: https://learn.ragdolldynamics.com/releases/2020.12.01/
> Several issues have been fixed to improve gameplay experience.
But at the same time I guess I can see why companies and project creators don't go into more detail. Maybe the actual bugs fixed were so weirdly technical that most users wouldn't understand them at all, or care (if the user hovers over this button for 1/300th for a second, there's a 0.0001% chance their next message might not save properly). Maybe they (perhaps sadly) don't remember what exact things they fixed, since they fixed so many bugs.
Either way some more detail would be nice for those that do care about this stuff.
On the flip-side, the changelogs for Super Smash Bros Ultimate and Splatoon 2 are some of the best I've ever seen.
All we can really put in the change log is bug fixes, and the circumstances that result in a bug can often be obscure and hard to describe in user-facing terms, so it gets summarized: "bug fixes and performance improvements". And for the play store and the app store, we have to put _something_.
If you miss an update, you'll miss the intervening changelog. There's no ability to provide multiple changelog entries in the app store.
This is a non-sequitur.
There are all kinds of apps in an "app store" context, and the apps I use serve me very well, thank you. I also don't generate any "content/engagement" for any of them.
Whether apps have good changelogs or not is also not a factor of whether the apps are in an app store or not.
You maybe meant to say "social apps" as opposed to "app store" apps.
For reference: https://docs.usertrack.net/changelog#usertrack-3-3-1-11-dece...
Every little task like this is a waste of time and mental energy that slows down useful work. In my opinion, this is just not worth the effort.
Many years ago I worked for a tools company where, for our less popular tools, user count would be in the thousands to tens of thousands, and for the more popular tools, it would be in the hundreds of thousands. Every single time we put out a release you could guarantee that a handful of people would ask for release notes even though we'd always publish them. Clearly people wanted them.
Now I work for a company where our platform caters to two very distinct groups of users - one group internal, the other external. Particularly for the internal users release notes are hugely important in order to avoid disrupting their work and deliverables to customers.
In both companies we've often had an "additional bug-fixes and performance improvements" item in the list.
Why?
Because people only have so much attention to go around: they're busy and, particularly in my current role and with our internal users, it's critical that we draw that limited supply of attention toward the most important changes. That's much more valuable than sending out an exhaustive document of every single change every time we do a release. We've found that if we supply exhaustive notes, fewer people tend to read them, and we get complaints that they're too long. As you've already suggested it comes down to ensuring that the release notes you provide are actually valuable to users.
Together with other documentation and training materials, release notes can provide a useful jumping off point for learning how to work with changes and improvements to our software.
(Not everybody reads them, of course, but having them does at least mean that when they inevitably ask for help/information, we can point them at the release notes as a starting point.)
It does depend on the product though. I can see a developer tool maybe even just going so far as to putting out their internal change log, but for other customer segments, it’s best to keep it short and (hopefully) funny
- why is performance often favored instead of improving, refactoring, documenting, simplifying, etc. And how can we change that?
- why are bug fixes so hard to track? Often developers fix something without marking it as a bug fix. And how can we change that?
(For many reasons, but including: "We added a new backdoor to satisfy government mass-surveillance program requirements.)
Most general users of apps on phones wouldn't give an absolute crap if you "fixed a buffer overrun in the X function", because that means nothing to them. If they read that you've made performance improvements, then great they can continue to play Candy Crush but not have it crash so often. They won't CARE that you "optimised the use of the strcopy function" or some crap like that.
Remember the app stores are targeted towards consumer users of phones, not tech people that live on Hacker News websites.
love 'bugfixes and performance improvements' in dutch
Usually the person who was a major contributor to the highlighted feature of the week writes the changelog and then we look through our PRs to add anything that could be interesting. We don’t include things like “migrated x” “upgraded library x” if there is no visible benefit for there.
Wrote about the lessons here: https://link.medium.com/9YuBV3dUIcb