Git-cliff – Generate changelog from the Git history
git-cliff.org
git-cliff.org
I have a program which generates the first draft of the rclone changelog from the first line of each git commit. I try to encourage all contributors to make the first line of their commit message be something a user would like to read in the changelog. Instead of `fixed nil pointer error` have `fixed crash when copying file to xyz backend`.
I also spend maybe an hour each release editing the auto generated changelog, removing the noise (refactored X, fixed docs for Y, made tests for Z work), condensing multiple entries, reorganizing, moving things to the correct section, linking stuff etc. Where the committer didn't write a sensible first line commit message I go back look at the diff and rewrite it.
I try to put the important things first in the changelog and keep it brief. The changelog is a heads-up for users that things have changed or been fixed and users don't want to spend hours reading it.
I think for an open source project like rclone where I review commits from all different levels of developer, and from all different levels of English mastery I'd have too much difficulty getting every commit message written in https://www.conventionalcommits.org/en/v1.0.0/ style to use a tool like git-cliff to generate the changelog without editing it.
Would I like not to spend an hour or more of my time editing the changelog for each release - most definitely! However I owe it to the users to make something nice and I don't think I can delegate that to a program.
> I try to put the important things first in the changelog and keep it brief. The changelog is a heads-up for users that things have changed or been fixed and users don't want to spend hours reading it.
Provided the commits use conventional commit format such as "feat(thing): fixed crash when copying file to xyz backend" or "fix(database): generate migrations correctly" not only do you get the type of commit but also the category or area.
git-cliff and others then use that to automatically add sections for each category of commit. This wouls surely needing to spend hours on writing changelogs, which seems an insane amount of time to me.
> I also spend maybe an hour each release editing the auto generated changelog, removing the noise (refactored X, fixed docs for Y, made tests for Z work), condensing multiple entries, reorganizing, moving things to the correct section, linking stuff etc.
This isn't easily discoverable from the landing page -- I think the site could benefit from an "examples" sections. Given the focus on customizability, it'd be good to get a sense of what's possible, rather than diving straight into Get Started, before I've decided that I want to use the tool.
I've found that storing changelog snippets in the repository is far more scalable (one file per note in a directory). This allows future changes to update changelog entries and reverts to just remove them completely.
There's also enough trouble getting Github and GitLab to support commenting on commit messages (something they should do anyways!) that feedback on release notes stuffed into commit messages is then a third-class process. They can still be collated and gathered just like from the commit messages, but they're actually tracked by git and there's no confusion when backporting changes: the release note comes with it without having to merge with conflict notes, `cherry-pick -x` notes, etc. You're also not limited to a measly single line for describing your change.
The things that conventional commits adds to commits were intended to be accomplished by commit trailers (Co-authored-by, Signed-off-by, etc.). Nobody is stopping anyone from having a “Change-type” trailer and a “Product-area” one too. You can have these in a template on your machine so it’s the same amount of typing involved. You can even tack on `--trailer` to `git commit` if you insist on using `-m` for everything and never filling out the commit body (commits devoid of detail are another plague on repositories).
There’s also more freedom to be verbose (“Change-For-Downstream-X”) since they are at the very bottom of the commit message. You can even filter out certain trailers with git log.
The Conventional Commits people were apparently not aware of this capability when they started (understandable since the tooling is niche). You can see they had some back and forths about trying to get their “footers” in line with trailers and how they had to stumble across misunderstandings about how they work.
Do you have an example repo that you can share? This sounds really ideal but wondering how this is managed and organized between releases.
So you still get authorial control over the changelog, but when it comes to actually creating the document it’s automated because you wrote the individual lines when you wrote the code.
It’s basically the same as what you’re doing, but in a different order.
Cue the Kubernetes release notes, which contain 20 separate "update to Go x.y.z" for each release.
A changelog is a raw, engineering driven manifest of everything that has changed. This log sometimes needs more context. Okay the wizbang had a hoozit added, what the hell does this mean and why do I care? Or it needs less and a lot of the engineer rambling needs to be stripped out.
The changelog is an input to a release notes process. This is where refine the wording and fix it up for end user consumption. Having a clear changelog pointing back to individual PRs and engineers lets us go back and talk to folks and clean things up and have final wording checked. There is no amount of automation that can every replace this process. It's part of the finishing touches and craftsmanship our customers expect.
Nothing I'm saying here is any different than what you are saying. My changelogs are the same as your drafts. I just found having those clear definitions and separate terms made it easier to sell the process and have people understand the big picture.
With that definition it literally is just the git log though isn't it, which was GP's point?
I know you're writing in agreement, but it sounds like your changelog is an intermediary step, and I don't understand why it's needed? To fill in the gaps from lazy commit messages?
I think you're right if you go by that definition ("changelog is a raw, engineering driven manifest of everything that has changed"), but there I think there is an intermediate step between git log and the user-facing release notes, whether or not it's formalized, which is to basically squash the messages into the list of effective changes since another point-in-time (eg: last release).
Let's say you add a feature, then refactor it a bit, then fix a bug in that new code. The only thing that needs to be there is the "Added a feature" part, because the rest is meaningless noise when you compare it to the prior release.
Likewise, if you add then revert something, both can be removed.
Or if you bump a dependency to version 2.0.5, then a couple days later go to 2.0.6, you don't need to mention the 2.0.5 one at all.
For me personally, this is just part of the process of how I go from git history and PRs to actual user-facing release notes (that have "why", context, and are sorted by added vs fixed, and have callouts of deprecations or breaking changes). But there's a case for it being its own output, or even the only output.
It's also great to ship that changelog with the artifacts so they are readable on third-parties (Helm Charts on Artifacthub, GitLab and GitHub releases, ...).
It's about the audience.
* Commit Messages - These are comments that last for the life of the project. The audience in for engineers far in the future and it should explain what changed, why and the context
* Pull Request Message - These are comments that live for a short time and are targeted for reviewers. Things you might cover here that aren't covered in a commit message is what kind of manual tests were done, releases you want to target, or reference other PRs that might be related
* Changelog entry - These audience here is internal. It is SREs, release managers and the authors of the release notes
* Release notes - This is for the customers
For many changes, all of these can be the exact same thing. It's one reason why I like conventional commits. In those cases you can write it once and it shows up everywhere with very little friction. You add it to the commit message, then when you open the PR the title and body are populated from the commit. When you generate changelogs, it is pulled from git history. Engineering isn't always the happy path, for special situations in each one of those steps you can edit it and add or remove context for the particular audience.
Not all orgs need this or need to care about it. It's completely okay if they don't. I've just found it useful in mine.
CC style can easily be enforced automatically, I often just rewrite commit messages when merging PRs from contributors, so it looks nice when I run git-cliff on it. You can always add a pre-face by hand to what was generated for every release.
> Using commit log diffs as changelogs is a bad idea: they're full of noise. Things like merge commits, commits with obscure titles, documentation changes, etc.
> The purpose of a commit is to document a step in the evolution of the source code. Some projects clean up commits, some don't.
> The purpose of a changelog entry is to document the noteworthy difference, often across multiple commits, to communicate them clearly to end users.
however. ended up just getting:
- fixed things. - creates new feature x - fixed broken thing.
instead of something that's appreciable by non technical people:
- when navigating to the nuclear launch code dashboard, a user is no longer mocked for having a likeness to <current unpopular person>.
point here is. if your team didn't write good squash merged PR titles before, they won't magically start doing so because you're using changesets.
From another angle, merge commits can also be a solution to the problem. `git merge --no-ff --edit` can be a great way to summarize an entire branch of commits. Most PR tools will give you an easy way to create those kind of merge commits. Don't settle for the default "merge branch x into y", create a meaningful title and fill in details/summary of what happened in the branch. With traversal tools like git log --first-parent you can see a high level of just your merge commits with the gnarly details of whatever steps led up to the merge commit itself.
I've certainly seen good projects where `git log --first-parent` was always a useful first pass changelog (no matter how "clean" the rest of commits were or were not). Probably still not a changelog you should send as a document to end users (because still written from a development standpoint rather than a user standpoint), but a good place to start writing the end user documentation.
The things that make this work well for us are
- We make sure we document every PR with user facing language and conventional commits.
- We generally use GitHub's squash merge, which means the noisy development commits are not part of the changelog.
- We create a highlights doc when we release that summarizes the main points of the changelog (this is the distinction between a changelog and release notes). E.g. https://ratatui.rs/highlights/v027/
It looks nice. And verbose. Like a lot of refactoring bullet points.
The “squashed” commits don’t look good but that’s the fault of squash commits (as usual).
In the end this is a very light shim on top of git log. (Light in terms of data, not light in terms of the discipline and visual overhead the so-called Conventional Commits standard demands).
Yeah pretty much. The config is at https://github.com/ratatui-org/ratatui/blob/main/cliff.toml
> It looks nice. And verbose. Like a lot of refactoring bullet points.
Most of the bullet points are single features / changes to the library that are delivered to the users of the library.
> The “squashed” commits don’t look good but that’s the fault of squash commits (as usual).
Can you give an example of this? Everything you're seeing in the changelog is a commit that has been pushed to the repo with a title message and a manually crafted body - we generally avoid letting the body be the github squash commit PR message junk.
> In the end this is a very light shim on top of git log. (Light in terms of data, not light in terms of the discipline and visual overhead the so-called Conventional Commits standard demands).
Yeah, and that's generally useful from the perspective that it's available online, searchable in a single doc (which makes it easy to answer questions like "when was blah implemented"
There's probably a bunch of things that would make this even better, but there's diminishing returns on the effort involved in doing so.
Hopefully this software is better but this seems like an opportune place to lodge this complaint.
This is not useful to decide if I should upgrade now, or what should I try out after upgrade --- and this is what a release note should be doing.
I disagree: I think it's extremely helpful for users to read dozens or even hundreds of commit messages saying "fixed typo".
/s
Normally, commits must match a particular pattern to be included. Those that start with "feat:" might appear in the features section, while those starting with "bug:" show up in fixes.
A few simple rules are enough to ensure that an intelligible changelog is generated. Minimal cleanup can then focus on making it presentable.
You could even set up your pull request rules to handle this for you. On approval it could look at the linked issue and include the title and appropriate pattern for that issue as the message for any merge/squash commits it creates.
>On approval it could look at the linked issue and include the title and appropriate pattern for that issue as the message for any merge/squash commits it creates.
Squashing is great IMO, but the problem is that many organizations prohibit it, because it erases a developer's commit history, because apparently it's really, really interesting to pore through dozens of commits where a developer fixed some whitespace, fixed some typos, etc. (Before you say something about rebasing, these same organizations also frequently prohibit that too.)
Making snarky comments isn't particularly interesting, nor does it offer much opportunity for discussing the actual benefits/drawbacks of the topic.
> Squashing is great IMO, but the problem is that many organizations prohibit it, because it erases a developer's commit history, because apparently it's really, really interesting to pore through dozens of commits where a developer fixed some whitespace, fixed some typos, etc. (Before you say something about rebasing, these same organizations also frequently prohibit that too.)
That's an issue with the policies of those specific organisations rather than with the idea of generating a changelog from commit messages.
Squashing/Rebasing isn't necessary to generate a clean changelog. You don't need to include every commit by default.
The idea behind auto-generating the changelog with a tool like git-cliff is to use it in conjunction with conventions on your commit messages like Conventional Commits [1]. This gives you fine grained control about what will appear in the changelog.
What you do, is to move the decision whether some information ought to appear in the changelog from the time of release to the time of the commit. You also shift the responsibility from the release manager to the individual developers. The big advantage of this process is that it is much harder to forget to include things.
If that is a good idea depends on your project and team. For open source projects, where everything is public anyways, the generated changelog could act as a pretty-printed, filtered and limited view of the git history.
For proprietary software I believe the commit messages should be a space where the developers can express themselves without having to worry about that anything they write could end up at the customer (which I presume is the ultimate consumer of the changelog).
I also think there is a difference between a changelog and release notes with the former not being a replacement for the latter. For a good example how in my opinion useful release notes could look like see [2].
Apart from all of that git-cliff is an excellent implementation.
Which doesn't make any sense.
20 commits/PRs might fit together in as one line in the changelog. There is nothing that one can write in any one of those commits that make any sense at the time of commiting.
Still, stuff like:
7a48c5b (cell) Add EMPTY and
(const) new method by @EdJoPaTo in #1143
This simplifies calls to
`Buffer::filled` in tests.
A better example, in my eyes, is teamcity.
https://www.jetbrains.com/help/teamcity/what-s-new-in-teamci...
A curated overview that actually explains the change rather than just states a difference.
They also have a similar view of fixed tickets: https://www.jetbrains.com/help/teamcity/teamcity-2024-03-rel...
Doing what teamcity does takes some effort. And I guess my gripe is the belief that these conventional commits will save the majority of work of the release notes. It can be a convenience but it needs curation and work to be presentable. And most projects seem to pick them to avoid that work.
My knee-jerk reaction is that now we’re just inventing something to sandwich in-between two already existing and useful things: the git log and the curated end-user change documentation.
Great work, git-cliff devs!
For an app to be useful you need to have a case where you want the data from the commit history to be useful in the context of a doc, where you retain technical descriptions written by devs. Just copying the commit descriptions into a doc, even in a nicer format, feels like theatre to me. You aren't gaining anything. Non-technical users won't benefit from it and everyone else can learn 'git log'.
There is an arguement that using this would push a dev team to improve their commits, but in my experience if the team lack enough discipline to write them well a time saving tool isn't enough to push them to get better. They're probably not writing a useful changelog yet, so this doesn't save any time at all.
I really hope I'm missing something.
I disagree. As a technical user who consumes many external repositories, I much prefer reading a changelog to reading a list of commits. First, I'm not reading this in a git client; I'm reading it in a web browser (likely on Github). In this interface, being able to see via headings the changes since my current version and the new version is extremely useful. Second, the change log is distributed with the application, even in cases where the source code isn't (this is especially useful when thinking about things like GUI auto-update checks, where I want to see the change log before I decide to upgrade). Third, the one-line summaries typically link back to commits or PRs, so when I see something I do want more information about, I can easily find the technical discussion about it.
It provides a solid, accurate draft that can be expanded on when targeting non-technical users.
I've often found that lack of visibility leads to messy commits. Surfacing these messages in the changelog introduces an incentive to take more care.
With regards to the usefulness of the descriptions, the associated issue is often linked along with the commit message. This is often omitted in hand-written changelogs. In this case the auto-generated changelog acts as an index, allowing the reader to quickly parse what changed and jump to the associated code or ticket.
> Just copying the commit descriptions into a doc, even in a nicer format, feels like theatre to me. You aren't gaining anything. Non-technical users won't benefit from it and everyone else can learn 'git log'.
> I really hope I'm missing something.
Yes, this seems to be cynical for the sake of cynicism. The point of tools like this is that you do in fact want this document, because no one will be copy pasting messages from git log to put in the changlog files. And if they are, they would appreciate a tool to do it for them.
Additionally, changelogs are part of releases and will show up in GitHub releases, package managers, etc.
Now you could argue that one should look at all dep code changes... But it's a tall order to fully audit all dependencies.
If you follow this properly the commits will already have important metadata for presentation in a changelog.
Also keep in mind that the auto-generated changelog should always be augmented and filtered manually before the final release.
But the combination of git-cliff and conventional commits can save a lot of time for the initial draft.
It does require discipline in using conventional commits properly.
This still requires discipline to choose the "right" type and scope.
[0] -- https://github.com/compilerla/conventional-pre-commit
There it goes! One more thing to do precisely and not only at dedicated time but continuously on top of all other things to do precisely already inherently around completing a task. Everyone all the time think about the state of changelog communication in addition, and do it well, phrase it for changelog purposes, format it for that as well. While working on changelog on the end will need additional work still.
In some simple situation it may have merit though, I admit. Seems like additional trouble (both workload and source of mistakes) in a medium to large team.
Except if the developers are well programmed robots themselves, of course, those do not fail (or easy to throw out and replace if they do).
Before someone starts getting their pants in a twist at this idea, you can configure a solution such that commits to branches don't need a conventional commit format, but that the commit that merges a PR into master must have conventional commit format.
People make mistakes! Everyone! Already.
More things to watch for and take care of, with more people: more mistakes.
Too much mistakes - that we already have without more aspects - negate the advantages of the idealized intended outcome.
People working with other people in everyday situations could understand that.
https://git-scm.com/docs/git-commit#Documentation/git-commit...
Not can, must. You cannot enforce this for local development since git(1) doesn’t allow project-wide hooks for security reasons.
So this is a way to reduce overhead, you need to write a commit message anyway so why not reuse that effort.
"Simple" projects like curl use their commit messages for the changelog.
Does it though? What would happen if we instead directed our discipline into updating the changelog manually on each PR?
Does having git cliff as the middleman provide much value if I have to enforce highly specific logging practices in the PR anyway?
We do monthly releases, and the level of effort has been under an hour each month per dev, which has been super worth it for us.
Being able to dedicate a tag, field, or work item type is pretty handy.
Ultimately we also decided to go with the tickets, but I am curious what your reasoning was to go that way?
The primary advantage of the ticket-based approach is that it's much easier to involve non-dev stakeholders. I'd choose it whenever other people might want input in the process. Most ticketing systems also offer a lot of flexibility, you could incorporate the ticket name, group tickets by relation, block completion states, act on deployments, etc. The ability to edit the note without rebasing is a major bonus as well.
The git-based approach potentially leads to a more readable commit history, and strongly associates any release notes with the actual code change. On the other hand, it's a pain to edit and can distract devs while they're problem solving if not setup well.
It works really well so far.
We also might need customer facing release notes at some point. But that shouldn’t be in the commit messages I think since it’s irrelevant in that technical context.
A big consideration for us was that we didn't want to get in the way of normal development, and Git is a core part of that loop. By making the release note a thing that happens separately, we let devs batch that work before a release. But the tool displays the commit right as you're writing the note, so you can quickly see what internal-facing commit message you wrote at the time, and copy-paste if you think it's good enough.
I wrote a tool to validate commits, which helps ensure both the git history and changelog look clean: https://github.com/welpo/git-sumi
Here's a better example on a more mature project: https://github.com/welpo/tabi/blob/main/CHANGELOG.md
It’s impossible to rewind your brain to the time of when the change is made to consider the total impact to the end user. The best time to write a changelog is when the change happens.
For minor stuff that doesn’t need to be in the changelog, the action checks a label and passes if “skip changelog” is applied. But the default is “hey, you forgot to tell users what they should expect with your change”
- The mandatory formatting takes up prime real estate in the subject line
- You don’t get much data out of it: one byte (rounded up) since it’s just “type” and “is breaking change or not”. Not a great trade considering how much it sticks out
- It subjectively looks bad: “feat”, “chore”, codey exclamation mark (punctuation) for breaking changes, and BREAKING CHANGE in all-caps (it’s supposed to be machine readable so why shout, your programs are supposed to pick up this for you)
- You have to care about this for every final commit that lands in the project
- Just to serve a changelog (which is supposed to just ape the git log?)
Same as having a standard coding style, you should have a standard commit message style that you do care about for every commit in the project. A mish-mash of variable quality and variable formatted commits more than subjectively looks bad - it reduces the signal-to-noise of the commit messages.
As for the `feat(scope):` in the subject line, it's a very succinct way to communicate the type of the change. Don't make me the read the whole commit to figure out if this was a feature or bug fix, or only affected the build.
> Just serve a changelog
Sure. But I want the changelog to list features first, then bug fixes, etc. How do I do that automatically with only the git log w/o some sort of standardized tagging of the commits?
I already personally use a standard commit message style.[1] And that standard is about how to structure the prose, not about using any kind of structured markup (outside the trailers section).
> As for the `feat(scope):` in the subject line, it's a very succinct way to communicate the type of the change. Don't make me the read the whole commit to figure out if this was a feature or bug fix, or only affected the build.
Verbs already serve that purpose most of the time. Then you get redundant lines like `feat: add OAuth 2.0 login`.
> Sure. But I want the changelog to list features first, then bug fixes, etc. How do I do that automatically with only the git log w/o some sort of standardized tagging of the commits?
If you have such a high volume of commits (that users would care about) that you have have distribute tagging of all commits to all contributors,[2] you can use the trailers section.[3]
[1] I don’t expect anyone else to at work. In one free-time project people use the same style, with one exception (against my personal taste).
[2] And if you need such a detailed changelog, let alone a fully automatically written one
I'm also aware of related tool in this space: semantic-release <https://github.com/semantic-release/semantic-release>. I haven't used it in my repositories, but it seems like a more comprehensive verison of git-cliff.
For everyone saying "manually written changelogs are so much better" - well, yeah of course! But they also take up a lot more time and resource to curate. I'd hate to see a huge project like gnome or something do this, but I don't think that's the pitch.
As an alternative for either a badly maintained changelog, or no changelog at all, adopting commit or PR conventions is a great idea.
That said, I am wanting a Github action based release workflow and want it to scale to manual edits of changelogs, so this is insufficient for me. Instead, I've been taking notes on changelog fragments as I hope they can offer the best of both worlds, see https://github.com/epage/epage.github.io/issues/23
But if you insist on using the git history for change log, why not use an LLM? You could feed git history to chatgpt and get a nice user-friendly change log.
https://en.wikipedia.org/wiki/Spoonerism
“The GIF search tool that finds it”, perhaps.
Because of this Windows users can simply run "winget install git-cliff".
Small things like this help with adoption.
I think there should be both a generated changelog and an official changelog when a release is made.
Git log ≠ ChangeLog