Ugh. One of my pet peeves is the generation of release notes from commit messages. Commit messages and PR descriptions have a different audience (i.e. contributors) from release notes (i.e. users).
For example, take a look at ESLint's autogenerated changelog [1]:
67c0074 Update: Suggest missing rule in flat config (fixes #14027) (#15074) (Nicholas C. Zakas)
cf34e5c Update: space-before-blocks ignore after switch colons (fixes #15082) (#15093) (Milos Djermanovic)
c9efb5f Fix: preserve formatting when rules are removed from disable directives (#15081) (Milos Djermanovic)
14a4739 Update: no-new-func rule catching eval case of MemberExpression (#14860) (Mojtaba Samimi)
7f2346b Docs: Update release blog post template (#15094) (Nicholas C. Zakas)
fabdf8a Chore: Remove target.all from Makefile.js (#15088) (Hirotaka Tagawa / wafuwafu13)
e3cd141 Sponsors: Sync README with website (ESLint Jenkins)
05d7140 Chore: document target global in Makefile.js (#15084) (Hirotaka Tagawa / wafuwafu13)
0a1a850 Update: include ruleId in error logs (fixes #15037) (#15053) (Ari Perkkiö)
47be800 Chore: test Property > .key with { a = 1 } pattern (fixes #14799) (#15072) (Milos Djermanovic)
a744dfa Docs: Update CLA info (#15058) (Brian Warner)
9fb0f70 Chore: fix bug report template (#15061) (Milos Djermanovic)
f87e199 Chore: Cleanup issue templates (#15039) (Nicholas C. Zakas)
I'm reading release notes to get a feel for how the new release might impact me. This takes so much time to scan, because there's so much useless cruft (to me, as a user) I have to ignore.
What's worked very well for me is to simply have an "I updated the changelog, if applicable" entry in my PR template checklist. Then when I cut a new release, I simply add the release date above the release notes currently listed under "Unreleased", and they'll list all relevant changes, reviewed during the pull request to verify that it is relevant to users.
I did explicitly call out that both commits and PRs have a different target audience (i.e. contributors) than release notes, because that is the root of the problem in my view.
PS. I do appreciate the effort btw! Better release notes in general is definitely something to strive for, and I hope you'll keep a tight pulse on how this affects what release notes on average look like and whether they get better, and of course how they can get even better :)
This is what got me so excited because I ensure my PRs have a user-friendly markdown Added/Fixed/Changed style. Can't wait to test this out for my next release.
It feels like a missed opportunity of more automation, TBH. Notice that each commit mentions one or more bugs, probably enforced by commit hooks.
What about dropping the ones that don't fix bugs (assuming they are part of ongoing work), replace the ones that fixes the bug with the description of the bug (and a working link!), and categorize them based on the tags of the bug (defect, feature request, etc...) so you would automatically get a list like:
New Features:
"title of feature request 1"
"title of feature request 2"
Bugs fixed:
"bug1"
"bug2"
The Release Notes are organized by PR and you can use configuration to sort into sections based on labels on those PRs. You can also ignore labels / particular users. So in theory everything you are outlining here is possible
> in theory everything you are outlining here is possible
I think part of the problem is also that in theory, it's already possible to write a good changelog as well — but of course, what's relevant is what happens in practice :)
Even then, in a commit message/PR description I (as a contributor) expect to read the origin/big picture of the bug, how it was discovered/narrowed down, references to where it was introduced, what other solutions were tried — those kinds of things. Those aren't relevant to users though.
This is why I believe Nate Graham does a fantastic job with weekly KDE updates[0]. He actually goes through the pain to condense every bugfix/feature addition to readable and grokkable single line, and it makes for great changelog read.
Of course, not every project can manage to write such a thoughtful post for every release, but if Mr. Graham can do it for something as sprawling as KDE, then can sure at least try.
> I'm reading release notes to get a feel for how the new release might impact me. This takes so much time to scan, because there's so much useless cruft (to me, as a user) I have to ignore.
so true, I even wrote a small script to drop the obvious cruft:
I guess so - I suppose contributors should be interested in the 'why', whereas users are more interested in the 'what' and, more importantly, 'how will this affect me?'
Oh I meant the opposite - a 'what' description (bad) being just an English re-run of the code in the commit ('add missing indentation') vs. 'why' that explains what problem it fixed/feature it added ('fix syntax error on navigating to page blah').
I meant that a good 'why' commit subject that gets propagated to release notes is more likely helpful to an end user - ooh nice yes I have had that problem, cool it's fixed, I don't care exactly what fixed it.
The "commit subject" should be a re-run of the code in the commit—condensed into single, 50-chars or less line. It's the rest of the commit message that should contain the detailed context, which includes the why, but most GitHub users are awful at both the commit summaries and writing the rest of the message. The "collaboration" that takes place on GitHub is bottom-of-the-barrel, and you shouldn't expect anything other than dregs there.
I don't think anything in the subject or message should be a re-run of the code, in the sense of converting it from programming language to English or whatever natural language.
Subject: Reason this commit exists ('why' e.g. 'Fix syntax error navigating to /about')
Message: Explanation for code, why it's correct fix, background etc. ('how' I suppose)
> I don't think anything in the subject or message should be a re-run of the code
Well, you're just wrong then. These aren't code comments. They're commit summaries. They exist only to answer the question "what?"--tersely. Because they're meant to help you pick out the right one when you're looking at a list of them and don't have the entire diff on the screen. But I don't think we're even in conflict here, because your example ("Fix syntax error[...]") actually shows this. I don't know why you keep calling that "why". It's "what". Any way of thinking of this as answering the question "why" (such as "why am I making this change?") is incidental. You're identifying which commit this is. That's a "what".
> Code: ... (that which I've called 'what')
What? No, code is never "what", whether you're talking about commits or you actually are working on the code and comments themselves. Code is "how", identifiers/names are "what", and comments and commit messages are "why". See my other reply to Vinnl about how weird it is to mix up "why" and "how" in the same breath.
The code is what I'm committing; the subject is why I'm committing it; the message is (a continuation of the subject's why and) a description of how I arrived at it, how it applies to the problem, etc.
Of course it can be phrased differently ('code is how I'm solving the problem; subject is what problem I'm solving' etc.) - I really don't think it matters much what we call them. Point is that, IMO, the subject especially and to a large extent the rest of the message should focus on the reason for the change, not re-describe the change itself, and framed in a way that's not necessarily end-user oriented but is certainly more detached from the implementation in code.
Less ambiguous than what/why/how then: code - the change; subject - the reason for it; message - the explanation of it.
Haha OK, I think we have different definitions of 'why' and 'what' - I was thinking of 'why' being "why did I solve this problem in this way" and 'what' being "what problem did I solve". So yes, totally agree, your last sentence sums it up nice.
Sure, it's a continuation of 'why', I called the subject that headed it 'why' after all. But a lengthier explanation of 'why' that describes 'how' the contained code applies to the problem I think can be called 'how'. I really don't think what we call them matters much if we agree on what they should contain.
Look at the amount of confusion that idiosyncratic terminology led to among the people here in this discussion alone. Of course it matters.
To repeat, first referring to the commit message body as why but then concluding you should call it how (because it can't be "why" since you've already decided that the summary is your "why" (which can't be "what" because you want the code to be your "what"))... is the kind of internal inconsistency that should be all the evidence you need to abandon this campaign to insist on using this terminology.
And speaking of inconsistency and insistence: if you truly think it doesn't matter, then there's no reason to insist on defending it. The only reason to do so would be if you didn't actually believe that it doesn't matter, and you just said it didn't because it seemed like a convenient way to argue that the weirder nomenclature is acceptable. But again, if it truly doesn't matter, then you'd have no problem letting go of that, right?
> you need to abandon this campaign to insist on using this terminology.
Sorry, who's insisting on it? I just said it doesn't matter what we call it, what terminology we use, just what we mean (and practise).
> argue that the weirder nomenclature is acceptable
I have no idea what you're talking about, taken as words alone they're ambiguous, yes, and can be used in different phrasings that make the single words apply to different things; as I demonstrated.
My expectations these days are so low, I'll settle for commit messages a full sentence, of any kind. I don't even get hung up about messages not being written in the imperative mood any more.
At least 'add missing indentation' is a few steps better than 'changes', or 'fixed', or dozens of commits with 'WIP' as the commit message.
Are you sure they list the individual commits? It looks like they summarize PRs when watching the demo screen capture. Will be interesting to see how useful it is for an open source project with diverse commit/PR message levels.
That's still a different target audience. Often my PR description will just be my commit message, though every now and then a PR consists of several commits and the PR description guides the reviewer through them and outlines a suggested way to review it, for example. Not at all what you'd want in release notes.
Its better than a project having no notes and trying to identify the range of commits manually, especially when there are breaking changes and trying to figure out how to update.
I agree - all commits to release notes doesn't seem very helpful.
We have a workflow that will only add to the release notes PRs that have the "Add to changelog" label which works okay, but sometimes you do have to decide who you want to address your PR title to.
In summary: Henceforth releasenotes will consist of mostly automated dependency bumps and the same commit messages that you can also see in the commit history.
…which is nice, when the master is not tagged and you’re trying to find out “does this release include the bug fix in pr #123” (without cloning and/or jumping between tags/branches).
Which is extremely easy: you click on the actual patch that has been merged and look under the commit message where all the tags containing this particular commit are shown.
Thank you. Tried on the commit in PR, did not show up, but the merge commit did.
I guess it's dependent on merge strategy, where squash merge will not include the initial commit hash. Or maybe "pr commit view" excludes the same info.
Nice. Shame the automatic changelog doesn't include commits that didn't go in via PR though. (Not possible to exclude locally rebased or squashed PRs I suppose, but all straight merges and anything done via GH UI surely covers most usage.)
My many year peeve is github adding "Source code (zip)" to the files in a release. It is the repository state at the tag, and needs some processing (think similar to generating configure etc), hence being a hindrance for someone looking for the source code to use. I do provide a useful source code file, but can't override github being unhelpful.
Do you plan on creating „real“ Github Actions actions to integrate with releases? All examples mentioned here use the github-script action, which, to me, feels like a workaround. Or some form of direct integration into Github Actions, e.g. configure a workflow as „pre-release“ and the tags and release notes are created automatically?
Given the currently documented automation features, I don’t see how this provides a tangible improvement to our team setups compared to current solutions like auto or semantic-release.
Automation like this is definitely a helpful first step for maintainers who want to prepare good release notes.
I wrote a provider-neutral guide for using git itself to help automate the preparation of good release notes, and also provide some more specific advice on how to make them worth reading:
Automatic change log generation is great, it's always a pain to do this. Hopefully the actions API makes it easy to output notes with better organization, like grouping conventional commits.
One more thing I'd love is a timeline entry/comment on relevant issues/pull requests to say that it was included in a specific release. I know we can do this via actions, but there's so many maintainers and consumers of smaller open source repos that would benefit.
I talked with one of the maintainers of conventional commits and we discussed writing an action to auto-label PRs based on conventional commits as a way to make a bridge between the two ways of working.
It would be nice if the release process also added the hash of any built binaries to a distributed append-only log. That's sort of the approach that sigstore and rekor were made for, to enable Binary Transparency, and they're already being used in the Arch Linux package ecosystem:
this is definitely on our radar! We didn't do any major work on artifacts / assets as part of this effort but there are a bunch of backlogged items, including individual hashes, that we would like to do in the future.
I think the fundamental issue plaguing virtually all forms of automatic release note generation boils down to the mismatch of audience. Commit messages and even pull request titles are tailored to be read and understood by other maintainers and contributors, who are likely to be familiar with internal details of the project. Clear and concise communication for this audience might mean using more specialized jargon or referencing implementation details.
By contrast, release notes are meant to be consumed by a much wider audience, who are chiefly concerned with the externally visible aspects of the project. Communication tailored to other maintainers (e.g. commits and PR titles) is rarely also optimal for this broader audience. This is why commit-based generated changelogs have a bad reputation: commits are meant for contributors—not customers—and thus tend to be useless.
This also explains why a good, dedicated CHANGELOG.md is usually so effective: unlike commits or PRs, it affords contribution authors a separate place to write for a broader, different audience. Another nice property of this method is the notes themselves within the CHANGELOG.md can be collaboratively reviewed and edited right within the context of the PR. This is a very helpful mechanism to ensure that release notes are high quality and to distribute the burden of writing release notes to those most familiar with the changes (as opposed to the person creating the release).
I think the ideal scenario is that on a per-PR basis, the "external" release notes are automatically scaffolded from existing metadata such as commits or PR titles (or even sophisticated means such as identifying which packages have changed or knowing if a breaking change has occurred as a result of type interface changes) which are later refined/edited during PR review. I think it would also be great if GitHub Releases themselves could go through a review process akin to PRs in order to help ensure high quality and facilitate collaborative writing of release notes.
> If you are automating releases with GitHub Actions, we have an API that you can use in your workflows to integrate the new Automated Release Notes feature right into your existing actions pipeline.
I can't seem to find the documentation about integrating this with Actions
This is fairly close to what we generate for our change log, closed issues and closed and merged pull requests keyed by release milestone. Release notes are separate though, and written by hand.
"GitHub is where developers come to learn and celebrate what’s new in open source, and where maintainers share, collaborate and celebrate their community’s work."
I thought GitHub was a closed-source SAAS by Microsoft.
For example, take a look at ESLint's autogenerated changelog [1]:
I'm reading release notes to get a feel for how the new release might impact me. This takes so much time to scan, because there's so much useless cruft (to me, as a user) I have to ignore.What's worked very well for me is to simply have an "I updated the changelog, if applicable" entry in my PR template checklist. Then when I cut a new release, I simply add the release date above the release notes currently listed under "Unreleased", and they'll list all relevant changes, reviewed during the pull request to verify that it is relevant to users.
[1] https://github.com/eslint/eslint/blob/master/CHANGELOG.md