Reformatting 100k Files at Google in 2011
laurent.le-brun.eu
laurent.le-brun.eu
The other correction I would make is that this post does not mention Nilton Volpato, who had written an earlier Buildifier and graciously accepted replacing his implementation with a new one and then taking over ownership for that new implementation as well. (Eventually ownership moved to Laurent's team.)
It looks like it was just under 2,000 commits. We did pretty extensive testing, by having Blaze load a BUILD file and its transitive closure and then dump that parsed form back out to a binary format. Any automated commit had to preserve that parsed-and-dumped binary format bit for bit. The slowest part of the testing was waiting for Blaze to do all the loads.
Every day I would prepare and test as many files as I could, break them into CLs (think PRs), mail Rob a shell script he could run to approve them all, and go to bed. Then I'd get up early in the morning (5am ET) to submit the changes, because there were various cached indexes that got updated when BUILD files got submitted, and it seemed better to send them when not many people would be working.
That scheme worked until a system did fall over and someone got paged, and then after that I agreed to only submit the large changes during business hours. :-)
Reminds me of all the impenetrable jargon around me when I was new
Rosie existed but very much wanted to break up the CL into independent per-directory CLs, and since I was editing one file in every directory in the entire tree, that would have been 200,000 independent CLs. I broke the list up by top-level directory or sub-directory and hit 100+ directories at a time.
Rosie also really wants to run each affected directory's tests, and I did not, because at scale flaky tests and such would be a significant source of false positives. The bit-for-bit check on the internal parsed representation of the meaning of the BUILD file proved that the changes were no-ops. That was better than any tests of the code in the directory.
I was already automating everything else, including deciding which files to change, reverting edits in files that were concurrently modified (they got swept into the next attempt), and the testing. Running a shell command to actually make the CLs was not difficult. And it generating the approval script trivial too.
Rosie is great but it wasn't the right tool for this job.
I also found this related quote from Russ Cox intriguing: "Most people think that we format Go code with gofmt to make code look nicer or to end debates among team members about program layout. But the most important reason for gofmt is that if an algorithm defines how Go source code is formatted, then programs, like goimports or gorename or go fix, can edit the source code more easily, without introducing spurious formatting changes when writing the code back. This helps you maintain code over time."
> end debates among team members about program layout
(Except in this case of course it's not "team members")
I'm definitely not getting this interpretation from the quote.
When I led the conversion of a decently-sized codebase from Flow-typed JS to TypeScript, I ensured that a code formatting tool that we were already using on pre-commit and CI called `prettier` was executed after each step. We took a git snapshot of each step of our automated conversion pipeline, and the diff was much clearer with `prettier` in place at each of those steps.
We've since used codemods frequently to make huge changes to the codebase in an automatic, reproducible, iterable way. They're very comfortable, very fun, and (thanks to the use of a formatter on all code) rarely produce incomprehensible diffs.
One thing I really love about gofmt is that it has no configuration at all. I think that was a major "innovation" and I'd love to see more languages adopt this approach.
I actually like how gofumpt formats stuff but ... nobody else on the team would have it, so it would make things worse.
I've never seen anyone with a workflow like this (lots of people have the second part, of course, but not the first one), nor tooling that makes it a really natural thing to do, but wouldn't it work? There are some pain points if you ever want to pair program or if you use multiple tools to collaborate on code.
If we ignore the fact that switching between those two formatters would "break" the formatting: There exist clean&smudge filters in Git, which could accomplish this technically.
https://git-scm.com/docs/gitattributes#_filter https://git-scm.com/book/en/v2/Customizing-Git-Git-Attribute...
By removing the maker’s marks, these tools make my code less readable. While, in my opinion, adding practically no value. I’m more than happy for every line of code to have consistent indentation (of course, but it did already). I also don't have a problem with silly but arbitrary formatting choices - like sorting my import lines. But these tools seem to drive so far for consistency that it costs readability.
That’s a nope for me. No debate.
If you're the tenth person, and you work alone, or with other fastidious people, you won't like the formatter. That's fine, you don't need to use it.
What about when someone leaves the company? Is it free game to reformat everything they wrote?
Why do you need to put your mark on code at work? It's not _your_ code. It belongs to the employer. The best work is work that is useful and not an irritant long after you're gone.
Sometimes it matters: indentation, naming_style and bracing should match throughout a codebase.
Sometimes it makes no difference: I really don't care about the order of your import statements. It simply doesn't need to be consistent throughout a program. It doesn't matter.
And sometimes making code "all formatted the same way" makes it all worse. I think thats true for spacing between functions. Functions simply shouldn't have the same spacing between them. Nor should lines of code within a function. Whitespace is a wonderful tool for telling the reader how lines of code group together. Gofmt erases all of that to make sure "code is formatted in the same way" - but in doing so, readability is actively decreased.
> If I move a function in a file with Person A's style to a file with Person B's style, do I reformat it?
Thats up to you! Why does everything have to have a right and a wrong answer? Obsessing over this stuff is a pointless waste of time. I guess thats the point of gofmt & friends - that you don't need to think about it. But, you can also not think about it by just not thinking about it, and letting your codebase be a bit inconsistent. Its not a crime. There are no consistency police. You won't go to jail.
> Why do you need to put your mark on code at work? It's not _your_ code.
You have an identifiable style whether you like it or not. Its evident in how you name your functions and variables. In how you write your comments, and where you put them. How you order functions, and where and when you split code between files, classes and modules.
Your style is inescapably everywhere in your work. And it will always have been written by you, long after you're gone.
Are you ashamed of how you write code? Why go out of your way to write and run tools that delete your mark on your work? It doesn't make the code better. Your team will not be more productive as a result. And it doesn't improve quarterly profits.
Like it or not, we're "creatives": That is, we're people who create. The software we write is distinctly our own. Having a little pride in our work is a very healthy thing.
There's no need for lengthy discussions with co-workers then. Just for bigger projects there's a need for Statement Macro Declarations, but that's arguably a bug/limitation in clang-format
Someday I want to study this and really understand what happened.
It really is dumb to be arguing over tabs vs spaces, after all.
You're right, ex. https://dev.to/jillesvangurp/comment/6gnb/
Sadly there's enough bitrot that ex. intentsoft.com is offline.
More here, but article is v opinionated/judgement oriented, comments are useful, but again, bitrot :( https://wiki.c2.com/?IntentionalProgramming
It strikes me that what I am describing as "bitrot" may also be "never really shipped, so the vagueness isn't accidental"
This always sounds more difficult on paper than just wrestling dependencies till dawn, upgrading from JDK 11 to JDK 17, for example. So I usually give up the mental exercise there.
Plus, following a file of transforms is mind-bending: someone may follow a method definition with a pattern seek, and then start appending some more code. Context is lost. It would be literate programming only with enough empathy for comments.
Which is all to say, would it be easier to move between Spring versions if the app's commit history were a series of transforms instead of changes to static files?
Suppose a commit establishes a framework version, and then follow a bunch of commits for domain objects, a skeleton controller, and so on. If we could play those decisions forward, but edit the transform instead of the source, would it be easier to dissect which next dependency to manage?
This loops back to ASTs: we would still edit and change files, but the history would be ed(1) macros (or something better, like ASTs). Somehow, it feels like there could be reconciliation between source control and "manipulating a timeline of changes."
Git may already have this, or a simple while loop with some decisions about how far to play the changes, like editing a cassette tape. A list of patches to apply, with pre- and post- hooks for rules scripts.
This saves you the trouble of authoring a separate programming language, or finding a way to preserve all of the niceties of the original syntax and formatting that wouldn't directly translate to an AST (like how many newlines are after a particular stanza or function).
Case in point: Recast (https://github.com/benjamn/recast) is of particular interest with regards to JS/TS in this vein, because it does preserve a lot of the spirit of the source in its conception of the AST. But also last time I used it (couple years ago now) it would explode on any code with an emoji in it. It's genuinely not an easy problem.
In an in-house dialect of Haskell I used to work with, we solved this problem by just making tabs a syntax error. Never had any problems.
(I think tabs might be have been allowed inside strings.)
(I don't really know what happened to it since.)
“We’ll just force everyone to do things one way” kind of ignores the point I was making. It shouldn’t be necessary for you to care how anyone else formats their code, same as you don’t care what font their code displays in or what text editor they use. It feels like a vestigial aspect of programming that we have to concern ourselves with it in 2024.
FWIW, just happy to have a chance to unload this thought finally: it had surprisingly little impact on code reviews, in that the "personal preference I need to enforce" just ascended abstraction levels.
And yes, that doesn't help you if ex. your style is a blank line following every code line.
In practice, it works, I surmise because people are fine with someone else's code being in a different style, but they want to write in their style.
Wasn't a problem in practice. Just like we never had any problems with anyone wanting to use eg Pascal or so.
In a language like C your outermost layers of leading whitespace are always indentation, and then you might have some alignment inside.
But in Haskell you might want to align arguments to a function, but some of the arguments can have blocks inside of them.
Mixing up tabs and spaces is technically possible, but it's too much of a pain in practice to bother.
More technically, language servers usually have a CST that they use to build the AST incrementally, and the AST contains references back to the CST that generated it. This is what allows you to handle incremental text edits and compile small deltas to the AST instead of the typical batch compiler design that attempts to parse everything all at once.
I've seen language server that completely ignores the parts with error, and I much prefer error nodes because then I still know there is something and these error node can still have children
There is also the problem that an error returned by a language server is a class of a "diagnostic" that includes syntax errors, semantic errors, warnings, lints, etc, associated with a span in the source code. It's much easier to think of diagnostics as a separate data structure that gets filled up during lexical/semantic analysis and associated with spans in the full syntax tree (you can even store them there as fields, but they don't necessarily have children). Then it's obvious how the structure gets created and fed back to the user.
And finally, the whole point of an AST is to be a valid canonical representation of a program so the compiler query it drives doesn't have to do additional input validation. So it just makes the queries/compiler passes easier to write.
On top of that, now you need to write a parser and compiler for your AST file. It's probably very simple and does rudimentary validation, but that defeats the point of the AST - it's a valid, canonical representation of a program by construction.
All in all, it seems like a good idea, and people have done it. But there's also good reasons to be apprehensive.
And at the end of the day, you need to ingest text as input, and you need to do it as fast as possible. There's not a ton of benefit to keeping an AST around on disk and in sync with the text that generated it when you are already able to compute it faster than you can read and deserialize it.
Blank lines is an obvious one: where do you insert them to "group" sections of a 30 line function? Or are there no blank lines at all?
Line length: just "wrap at columns X" (or never wrap) is not enough, because people can and do wrap at specific locations for specific reasons, because that makes more semantic sense or looks nicer than cramming as much as possible.
Whitespace gives readers subtle information about the structure of a function before they read any of it. Its a powerful tool. Dismissing or - worse - deleting whitespace wholesale sounds profoundly misguided to me.
Why make code harder to read? Where's the benefit to your approach? I can't see any.
https://doc.rust-lang.org/src/core/slice/mod.rs.html#2786-28...
The function is pretty short - 40 lines including comments. Despite how short the function is, it still uses whitespace to separate and group adjacent lines of code. Personally, I find the code more readable like this. Indentation makes syntactic blocks obvious (the while loop and if statement). But there are also conceptual groupings between lines that mean nothing to the compiler, but are semantically meaningful to humans. I can tell at a glance that the comment about safety is most associated with that one line below the comment. And so on.
I think this code would be worse if we deleted the whitespace. How would you improve this code?
At a minimum, the blank lines implicitly scope the comments that sit above connected code blocks. Those comments - especially the comment about safety - would be harder to understand and audit without its context being so clear.
And again, can you name any benefit to removing all the blank lines? If we can both read the code easily with some empty lines to space it out, that seems like the best option.
It works for go because gofmt was there from the start, so even if you are returning a multi-dimension array and elements come out unaligned, that's just accepted as how it is and nobody cares. For other languages, people will have to either accept "not caring" as becoming the norm, or actively fight the autoformatter from steamrolling over their code.
For people who would give more thought to how their code would be read, autoformatters were often more frustrating than "nice".
It's why we can't have nice things... Autoformatters do help though.
IMHO autoformatters are awesome until there's something you care about that the implementers didn't care about, then they are horrible. Problem is, the people who put a lot of thought into those decisions are often in the minority and tend to lose the argument.
https://help.eclipse.org/latest/index.jsp?topic=%2Forg.eclip...
The purpose of global approvers was exactly things like this. If you want to do a mechanical change to an insanely huge number of files, they can potentially approve it.
In my experience, global approvers were used extremely rarely, only in cases like this where the transformation was purely mechanical and it was possible to verify that there were no logic changes.
Most of the time rather than global approvers, you were encouraged to use a system that would automatically split your change into a bunch of smaller CLs (PRs), automatically send those to owners of each module, then automatically merge the changes if approved. It would even nag owners daily to please review. If you had trouble getting approval for some files you could escalate to owners of a parent directory, but it'd rarely be necessary to go all the way up to global approvers.
Basically if there was even the slightest chance that your change could break something, it's always safer to ask individual code owners to approve the change.
Typically, code authors would create a proposal by filling out a doc template. It's usually light weight and also accompanied with examples or full set of the pending code changes. Then 1-3 of us will review and LGTM the proposal. As part of the review, we also determine whether the changes should be sent to local code owners, or "globally approved" by one of us. The default option is to use "global approval", unless the changes need local code owner's knowledge during the code review. Said in another way, when sent to local code owners, their role is not gate keeping the changes, but to provide necessary local knowledge where we as global approvers don't have.
Refactoring changes, such as formatting or API migrations, shouldn't bother local code owners because 1) it would just be a waste of their time to review and approve; 2) in practice, we find a central code reviewer for the same large set of code changes is more likely to catch bugs (with review automation tooling) than local reviewers.
We consider ourselves as facilitators rather than approvers or gatekeepers of the code changes. Our goal is to make these changes done more efficiently and save engineering time when possible.
If you like stats: over the past 5 years, I have reviewed ~300 such proposals and ~40K changelists (equivalent to PRs). One changelist/PR typically contains 10s to 100s of files depending on the nature of the change. When I was most active, I was about ~5th-ish when ranking the number of changes we were approving. There are many global approvers who have approved more than 100K changelists, which is a milestone we celebrate with a cake. Too bad I didn't have the chance to have my cake.
Examples of global changes include:
- changes to Buildifier that require updating existing files
- rename/refactor a function used everywhere in the repository
- fix the existing code before turning a lint warning into an error
- fix code that will break with a compiler update
Anyone in the company can propose this kind of change. The proposal will be reviewed by a committee (to ensure the change is worthwhile, that are mechanisms to prevent regressions, etc.) and by a domain expert (the team that owns the area).
Global approvers are people who often deal with this kind of changes. They usually come from the language teams (e.g. I knew the specificities that come with global changes touching BUILD/Starlark files).
New approvers are nominated by an existing member and then LGTM'ed by other three. Usually they have gained a lot of large scale change experiences on the other side, and we recognize that we could use more help on the committee side. Especially we want a good coverage on various languages, tech stacks, and time zones.
But where you are nearly the sole owner of a small library and you are crafting that library to be beautiful and understandable... there is something pleasurable about structuring concepts so you have each on a single line, or creating similar functions so the concepts are structured by column.
I know not everyone will hold this view and that is fine, but when you are writing your own hobby library in your favorite language for your own purposes I recommend you try it out.
I guess I've never gotten into an actual serious debate with someone over formatting so I don't know what I'm avoiding, but sometimes auto formatted code makes it harder to skim
Once, a product launch depended on me urgently kludging a device driver in Python (long story). And this involved a large hand-maintained mapping table. I wrote it quickly but carefully, and found some formatting that made the table readable enough, without implementing a minilanguage in Python.
But the Black formatter had been rigged to run automatically on commit, so... poof! :)
https://black.readthedocs.io/en/stable/usage_and_configurati...
Other formatters have similar functionality; e.g.:
- /* prettier-ignore */: https://prettier.io/docs/en/ignore.html#javascript
- #[rustfmt::skip]: https://github.com/rust-lang/rustfmt?tab=readme-ov-file#tips
As soon as anyone else looks at your code or wants to participate in the development process, what is meaningful to you about the arrangement simply won't translate into their view; what is the use of a programming language if not to use its existing definitions and concepts to communicate the concepts of programming?
You can use fancy syntax tricks, but domain-specific languages are a much better way to handle the same problem. You can express things with them that other humans can understand while still retaining access to your existing formatting tools.
Considering I get pleasure from formatting the code beautifully I'm not sure how it's even possible to be doing _myself_ a disservice. It's like saying I'm doing a disservice learning guitar instead of listening to Coldplay on my speakers.
As far as others are concerned... I agree somewhat, but I don't think it's by any means proven. I think we are always better off then cobbled together code where formatting hasn't been thought about. By well-designed hand-crafted formatting can be expressive in it's own way IMO
I do want to say that I have the opposite view. People who use formatters want their code to be consistent and go the extra mile it ensure it does. It's like manual testing vs automated testing to me. Sure with manual testing you can test many more corner cases as they come up as an intelligent person is in the loop. But there will be mistakes made, tests forgotten etc. Just like there will always be inconsistencies when you manually format the code.
Many people who use formatters (myself included) just want consistent code and don't want to bicker with others about it. When it is soley owned by me and I'm doing it for fun, these reasons fall away for me.
I want to write syntacially-valid code, without worrying about the visual presentation of it. (I want a good presentation, but I don't want to put forth the effort to create it.)
And my IDE makes it very easy to indent a snippet.
Enabling tools like these was exactly the point of the enforced formatting. It worked extremely well.
For example: https://github.com/Calsign/gazelle_rust
Unfortunately, the flip side of this coin is harder to deal with; sometimes truly important issues or things that people do deeply care about have disproportionately too little verbiage. Finding those can be very difficult.
The meta says that it is. there are only 81600 seconds most days, and you get to choose them how you want, so choose how you spend them wisely. if that's arguing over tabs or spaces, then that's your choice.
This is basically the same claim that economics can treat humans as perfectly rational actors perfectly rationally pursuing their perfectly rational goals. It is not a good model of humanity.
"All models are wrong, but some are useful." -G. Box
(I think Go was already installed on every Google developer machine at that point anyway.)
But back to the point... formatting rules without firm, incredibly strict enforcement ends up being a tax on the janitors - the people who clean the code base and do large scale changes. That makes me sad. These are the people who care a great deal about code health, and their work is hindered by the lint checks that we have imposed.
Let me give an example.
I'm trying to eliminate a constraint in the build system. It's a "small" large change - only O(30K) instances. (Yeah, Google scale is different). I have an incredible wealth of tools available to me to automate the process. For the benefit of the Googlers, I can identify Blaze targets to change, use buildozer to fix them, and ship off CLs to review. But the changes I want to make are often ones which should be reviewed by the code owners, and not globally approved. So possibly O(10K) individuals might be involved in reviews.
Let's explore the problem. First, shouts to y2mango for bringing up incremental formatters. This should be the default for all tools. And another to flymasterv for raising the question of "why not just format as each person touches a BUILD file". Here's the situation.
1. buildozer is really good at rewriting BUILD files syntactically correctly. 2. It has an unfortunate side effect of not being incremental. It calls buildifier to rewrite the entire file. 3. We update the formatting rules to make them stricter over time. That means that a "correct" BUILD file on January 1, might require changes on March 1. 4. Buildifier findings are advisory, rather than mandatory. 5. No team is staffed with repeating the monumental work this post started with.
The reality on the ground is that little touched BUILD files become stale, and would require a formatting update over time. It is actually worse than that, because many teams take the path of ignoring buildifier warnings and committing their working code anyway. Without continual BUILD file reformatting there is a lot of stale floating around. [Root cause: We could fix this by promoting people for doing that repeat work. But we don't. We promote for the initial sprint.]
And then a janitor comes along.
I use bulldozer to fix a problem. It reformats an ancient BUILD file completely (not incremental). I send it to the code owner. They see changes far beyond my 2 line fix. They reject it, or ask for a change to only the two lines that actually mattered. Sure. I can hand build the change once or twice. But not for a few hundred, or thousands files. So.... I have to hack up an incremental format. Or, it turns out that users are very happy if I don't bother with formatting at all, and just change single lines. It's not that any individual is right or wrong. It is that they all have a choice and a preference and Google created a policy that allowed individual teams to have a choice of strict compliance or not. That is the failure.
If you are going to have a policy about code formatting: - make it hard mandatory for everything except a "break glass" situation - if the policy can evolve, staff a team with enforcing it globally
The fact that Google, as a company, does not reward this behavior does not take away from any individual's accomplishments. This post may sound grumpy to an outsider, but I am constantly amazed at the tools I have available to fix things on an enormous scale. The friction is usually only where we have good intentions, without the policy teeth to enforce alignment with the intentions. That's a management problem, not a technical one.
Auto-format those O(30k) files and get global approval. Then, separately, make your two-line semantic change and seek approval from local owners.
Wait! I thought Google only promoted and rewarded people who do new development, not maintenance. What gives?
> [Root cause: We could fix this by promoting people for doing that repeat work. But we don't. We promote for the initial sprint.]
If that means it's too hard to change the format rules, then don't change the format rules. And if you don't reformat, then it has to be a clear rule known to everyone (or written down somewhere you can point to) that incidental formatting changes are acceptable and not something you are allowed to push back on.
I can speak to Go and gofmt, and there we are VERY reluctant to change formatting rules. It does happen for the odd corner case once in a while, but nothing that would cause "changes far beyond my 2-line fix".
I keep a quick little scriptlet in my bookmark bar for cases like this:
javascript:(function(){ $('head').append('<style>*{color:#101010 !important; background:#f0f0f0 !important;}</style>'); }());
javascript:(function() { for (var n of document.querySelectorAll('a, p, li, div')) { n.style.color = (n.nodeName == 'A' ? 'LinkText' : 'CanvasText'); n.style.backgroundColor = 'Canvas'; n.style.font = '500 16px/1.4em sans-serif'; }})();
I was surprised at how helpful forcing the font face and spacing is. There's a lot of sites out there with bad-looking fonts or huge line spacing on top of unreadably-light gray.
I added the background color part based on your version. Thanks for prompting me to try that; the way my bookmarklet didn't work on black backgrounds was occasionally a problem. I also added a bit to force link colors, since neither of our versions handled those well.
Perhaps the next step is a "multistage" bookmarklet that applies more rules the more times you click on it, so the more forceful rules (like background color, which often messes up other parts of the site design) can be optional.
I tend to prefer interfaces in dark mode and content in light mode, so I'll see how I feel about the conditional logic there, I may eventually wind up going back to hardcoding some colours.
> Perhaps the next step is a "multistage" bookmarklet that applies more rules the more times you click on it, so the more forceful rules (like background color, which often messes up other parts of the site design) can be optional.
That's a really neat idea. I can imagine it stretching from slight readability tweaks all the way to a pseudo-reader mode. It would definitely be a bit more of an undertaking than either of our quick snippets, though.
The source code is text thing seems like a "legacy" concept.
At the very least I think this is an interesting thing to explore. Maybe it leads nowhere...
I have long felt that Google’s strength has always been making a bad architectural choice and then executing on it flawlessly. So many systems are designed in ways that require incredible technical execution to make them workable, and they do it.
And Google uses P4(ish) because monorepo, so they build further abstractions over P4 to enable git and hg in user space which erases most of the potential benefits of either which is also all really, really good software, but it’s all effort necessitated by monorepo. CitC is a work of art, but it is also something necessitated by a stack of other choices that forced their hand into inventing something miraculous to keep hacking around a previous limitation that nobody else has.
That seems like way more complexity than just doing it once and for all. Now the commit log is littered by a bunch of automatic commits that format one file at a time.
Putting the time of submitting the changes on a small team (mostly me, with approvals from Rob and help from Laurent) was absolutely the right tradeoff. It avoided the "unfunded mandate" and tech debt of making everyone else deal with it.
Update: I found the FAQ we wrote back then. It was very short. These were the last two questions:
Q: Who will update all the existing BUILD files?
A: We will. There are nearly 200,000 of them, and we’ll take care of that. We’re sending CLs out now. If you want to do it yourself, that’s fine: see go/buildifiernow for a tool that can help.
Q: You’re creating a lot more work for me.
A: We are creating significant amounts of work for ourselves, including reformatting all 193,000 BUILD files in google3. For the rest of the engineers in the company, we intend to make the transition as smooth as possible, with integration in Eclipse, Emacs, and Vim, as well as tools like Rosie and GenJsDeps. It is an explicit goal not to create significant work for other engineers. If, as we roll this out, you find that we’ve created noticeable work in your workflow, please let us know so that we can address that.
Your suggestion would allow people to bypass the code review by just saying "oh it's just cleanup don't worry".
I've always been against "reformat the whole code base" but it's an interesting example where it seems to have been the right choice.
Then, there are also other formatters that support "incremental formatting", meaning it only formats lines that are changed in your commit.
Disclaimer: I authored https://github.com/google/pyink and replaced Google Python's YAPF formatter with this Black fork and also implemented the "incremental formatting" feature in Pyink and upstreamed to Black.
When we were rolling out the formatter change, we chose to NOT format the Python files mainly because 1) not all teams at Google enforce Python formatting at presubmit time; 2) the formatter supports "incremental formatting" to minimize the diffs introduced by the formatter.
There are of course less ideal cases where even incremental formatting has to touch not-changed-lines, such as a large Python dictionary/list/set literal that spans across dozens or even hundreds of lines. It's a tradeoff in the end.