You should absolutely strive to keep smaller PRs, but I've frequently seen this become "you should only have small PRs."
Not sure I understand the "artificial" part here. There's nothing "artificial" about breaking up your larger changes into smaller PRs. It's just good practice.
Helps reviewers who are reviewing the code, and helps the author be more focused with their changes.
Even in net new feature development it's a good idea to break up your large changes to something more manageable.
Sorry if I'm not understanding, what do you believe the downside to be?
Often you need the full context when evaluating a new feature end to end.
Or you spend 2 days splitting up a PR into smaller PRs so that a person can review it in 30 minutes instead of 2 hours.
I can't say I've ever seen benefit from it both as a reviewer or as a developer, but it could be an effect of different companies and different teams.
I'd agree keeping the unified context is preferable but it's probably easier to do that by having developers rebase their changes into discrete commits that can be reviewed one by one.
Let's give a hypothetical situation where you split up a PR by the backend and frontend components, but you have some extra fields and endpoints that are not used in your final product. They accidentally get left in because you miss them.
I believe the chance that it would be caught in review is significantly higher in a unified PR review instead of artificially splitting front and back into separate PRs.
And if you need to have the 2 PRs open side by side, then why split it up in the first place?
I agree however I think PR's should be split up. Just not into separate PRs. The solution is to actually start reviewing PRs by commit (you can do this in the PR web interface). Everything is split up and can be reviewed separately but you can still see the final diff and while each discrete unit is reviewable, the greater feature is also still one discrete reviewable item.
Organizing by clean commits is definitely important.
I just grabbed a random patchset off of the git mailing list and linked it below [2] to demonstrate this.
You'll notice that on top of the overall patchset (equivalent to a PR) having a detailed description (in the coverletter, i.e. PATCH 00/xx), each commit has a descriptive name, message detailing what the changes within do, and signoffs by everyone who contributed to it.
Then as each reviewer comes in, they review each individual patch (i.e. commit) separately, replying to the message for that patch. And any high level concerns can be in reply to the coverletter (addressing the entire patchset).
As the contributor responds to comments and makes the requested changes, those changes get made per patch/commit via rebasing rather than just added on top. And when they are finished making the revisions, a new v2 patchset is released in reply to the cover letter of the first patchset, now also containing a diff per commit/patch against the previous revision.
Then the cycle repeats until everyone is happy. At that point the maintainer will merge the changes into their incubation branch (the git devs call theirs `next`) and after some time has passed, they merge it into master/main and it becomes established history.
The worst part of the whole workflow is that it uses email but other than that it is a far preferable reviewing experience to using github's stock pull request workflow.
2. https://lore.kernel.org/git/20231010123847.2777056-1-christi...
How would you know what to critically evaluate in such case?
If you are talking about refactoring prior to merging into the tree? then no it's not a waste of time. That's the intended workflow. You make your changes to the commits or add new commits in between using rebase. This is how all the development for linux is done.
If you are talking about after they are merged into the mainline? That's also not a waste of time. You don't have to go back and change those commits because they are now finalised and your iterative refactoring can be done via small one off patches gradually merged into main, a series of large patches merged into main, or a set of patch series all merged into an incubation branch that is kept up with main and eventually merged back into main once the refactor has made sufficient progress to take over.
1. I open a branch.
2. I start coding.
3. During coding I do various commits.
4. I realise I need to refactor/rethink something, I do more commits, overriding the code/ideas of previous commits.
5. Then I finally put it up for code review. But in such case there's no point in going through all commits, since a lot of that gets changed anyhow.
6. Usually I would squash everything into a single commit and merge after doing that. Because a lot of my commits would've been pointless anyway.
What is the suggestion exactly?
1. I open a branch. i.e. `ID-XXXX-branch_name-working`.
2. I start coding.
3. I make a ton of quick tiny commits. (I generally label these commits "NO-MERGE: ").
4. I make a bunch of changes.
5. I finally get everything done.
6. I now create a new branch `ID-XXXX-branch_name` from `ID-XXXX-branch_name-working`.
7. I rebase that branch to get my code cleanly formatted by commit with each discrete feature or change getting its own commit.
8. My code goes up for review.
9. I get changes requested.
10. I make a new branch `ID-XXXX-branch_name-v2-working` from `ID-XXXX-branch_name`.
11. I make the requested changes as a bunch of new small "NO-MERGE: " commits.
12. I am now ready for re-review.
13. I now create a new branch `ID-XXXX-branch_name-v2` from `ID-XXXX-branch_name-v2-working`.
14. I rebase those NO-MERGE changes into my "presentable" commits, adding or removing well documented commits as necessary.
15. I now send out my v2 revision to the mailing list or I change the HEAD of my PR from `ID-XXXX-branch_name` to `ID-XXXX-branch_name-v2`. If I'm using a PR workflow, I link a diff between the two revision branches (you can do this in github using `https://github.com/org/repo/compare`). That isn't necessary with patchsets since I can easily do a range diff there. I suppose I could do a cover letter with range diff and paste it to github but it somehow doesn't seem as nice.
16. Rinse repeat steps 8-15 as necessary for each new revision.
17. "LGTM"
18. Merge into `main`/`master` (like actual merge, not rebase or squash merge) and close PR.
19. Clean up branches. Either save them somewhere for prosperity if you have trust issues like me or just delete them.
This looks like a lot but I was trying to be as detailed about that workflow as I can be. Realistically it's not so bad and you can get a hold of it very quickly.
Also with regards to rebasing changes into well documented, discrete commits, if when you are doing your development you make your commits small and self contained, with `git rebase -i` you can actually just reorder the list of commits to chunk together the related commits and 99% of the time it'll rebase with little to no merge conflicts. Then you can just squash those chunks down into your presentable commits. This also applies to your v2 changes and on. You can just move those commits in the rebase TODO to put them after the commit you want to squash/fixup them into and if they are small clean changes, they should rebase without conflict. Things only get nasty and break when your commits are spanning multiple unrelated files and you try to break those up.
I'd estimate rebasing new changes into an already documented set of commits probably takes me 1-2 minutes on average so I consider it well worth the extra 30 minutes spent over the course of the week.
In my experience, this kind of thing happens much more frequently with large PRs.
If a PR is large enough, things will slip through the cracks. If there are too many change requests, even if they're sensible and make sense, things will slip through the cracks. If there's the need for multiple checks by the reviewer, even more things will slip through the cracks.
Also, breaking up PRs in frontend vs backend is not the best idea. Build the feature iteratively if possible.
EDIT: Since the grandparent is talking about reviewing by commit: if those are more reviewable, just use those as the way to separate PRs instead, maybe?
If a PR introduces a new function that will be used in the next commit, I would much rather see that next commit using git, than hunt for it in the PR queue.
But as a code writer myself, for example, if I am building a new feature, firstly it's really hard for me to know what the whole thing would look like without going through it all and it's probably very iterative process as I'm doing it, so I usually wouldn't be able to split it up or it would very suboptimal to split it up before I've finished everything.
Then I would try to split it up as I've finished to appease reviewers, but again, it requires whole lot of creativity to do. Should I try to split up shared component first? Because I surely can't split up whatever is using those shared components. If I do then, people won't see whatever is implementing those shared components so they won't have understanding on why those shared components provide certain functionality etc.
Overall it complicates a lot it seems because if I was to do it during my iterative process then I would write a PR, later refactor bunch of it anyway, and I would do it in the order that feels best for me, but wouldn't necessarily be easy to understand for anyone not within it.
To make it work, you have to think differently and code differently. You have to think in advance: "does what I want to do require changing a lot?" If so, think about how else you can solve your problem. Or when implementing something for the first time, think about how difficult it would be for someone to change it.
You end up building things with high cohesion, low coupling, object factories, etc. It makes for very different code that is more maintainable, flexible, easier to change. You end up not needing a big PR, or the changes are to a single high-cohesion component so it's much easier to review than changing 10 different components.
With that said, using methods like stacking and feature flagging, I've been finding even new feature development to be possible while keeping my PRs to roughly 5 files changed or less.