When it comes to Git history, less is more
brennan.io
brennan.io
If have commits like this, add the ids to a file `ignorerevs`, and then tell git about it:
git config --local blame.ignoreRevsFile ignorerevs
Then at least `git blame` will still give useful results. (This is a relatively new git feature, added a year or two ago.)https://stackoverflow.com/questions/58042532/how-can-i-clang...
But wouldn’t the standard -w be enough to ignore most of this commit.
But I used ignoreRevs in repositories that messed up their history by using an automatic indenter with atrocious settings.
Hm, especially for the case of blaming (mostly to figure out why something was done a certain way) I frequently happen upon 10+ year old commits. This happens both in open-source projects, as well as at work (where we don't use git, but the same concepts apply).
total = 0
for value in list:
total += value
return total
vs. total = 0
for value in list:
total += value
return totalStill need to specify the file yourself, so hopefully someone can point out the missing magic to have git pull in .git-blame-ignore-revs on its own.
[0] https://git-scm.com/docs/git-config#Documentation/git-config...
banana_giraffe was asking for a way to set it up automatically so users don't have to do anything.
That says to list all the annoying commits in a file, then use a new-ish (Git version 2.23) Git feature to ignore all those commits when doing git blame.
Notice that one of the comments in that article says how to get everyone to use that ignore file, when using GitLens with VSCode.
The real problem here is that “git blame” is a garbage tool. Perforce has “timelapse view” that is radically better than any historical view I’ve seen in Git/Hg.
With respect to this issue, git just needs to suck less.
My company has a web tool for browsing the repo that allows this. It’s still inferior to Perforce timelapse view. But it’s not a hard blocker.
git blame WS_COMMIT^ -- FILE
You’ll need to change the file name, though, if it’s been renamed since the whitespace change.On the other hand, I don’t think that the conclusions of this story are as inevitable as it makes them seem. Much of the pain it describes is due to the fact that Git is, as it proudly itself, stupid: it doesn’t understand that your spacing or line breaking or bracketing changes are incidental to what the code is trying to accomplish. I’m not saying that structural or otherwise language-aware editing or source control is the silver bullet because I’m well aware that so far most attempts at it suck, but I think it’s important to remember that whitespace pedantry is to a large extent a tooling issue.
Could a merge tool coupled less than completely to clang-format (or yapf, or black, or gofmt ...) use its settings to improve the merges, I wonder?
(Tangentially related: Monticello, Paredit, the “skeleton syntax tree” idea from Dylan.)
Taking the time and deep breath to write nice interfaces rather than scrambling and putting all the effort into implementations ignoring the larger picture of how human endeavors compose seems very fundamental. It reminds me also of the slogan
> A developed country is not a place where the poor have cars. It's where the rich use public transportation.
This is brilliant, stealing it for my transit advocacy.
- Text (or binary) streams are the ultimate unnormalized representation, and you don’t always want your data to be normalized. When storing it, maybe (though I’m sure the DBAs among us will disagree even on this point), but the intermediate values during processing, almost certainly not.
- Most representations of structured values are all or nothing, whereas in reality most types of structured data are usually more conveniently viewed as having a hierarchy of supertypes, possibly with things like “chunk of text” on the top, maybe “list of chunks of text” below that, and so on; otherwise your tools are much less reusable. (Compare “generic programming” in the Haskell/Scala not Java/C# sense, i.e. metaprogramming over algebraic datatypes not parametric polymorphism. See also the “nanopass” approach to compilers.)
These two (largely overlapping) points are how I explain to myself the (empirically if not theoretically evident) advantage of Unix’s “getta byte” approach to IPC (as Cutler sarcastically put it) over the myriads of structured approaches that came before and after. By not structuring your data exchange, it doesn’t force you into a framework that’s more rigid than it needs to be. People barely understand how to make structured data amenable to processing by composable tools now, back in the 80s I don’t think anybody even knew to ask the question.
This is why I mentioned Paredit and Dylan’s “skeleton syntax trees”: they are, respectively, a structural editor and a macro system that work where others fail by the virtue of being expressed in terms of more than text or even tokens, but less than a full syntax tree.
Finally, in this specific case,
- We’re talking about programming languages, about making a computer understand the programmer’s intent to some degree. This is a maze where making even the slightest wrong turn has Gödel and Turing together slapping you across the face with a giant NOPE sign. It’s kind of endearing to see the luminaries on the Algol committee (essentially all of CS at the time) state that they want, like, pseudocode, from their papers, except, y’know, with a compiler, and not suspect anything can go wrong with that. But we are hardly smarter than them, so we should probably remember the bitter lessons they’ve been forced to learn and keep in mind the possibility that there might not be a good solution to our problem.
I think "normalized" is doing a lot of work in this sentence. In the context of data keeping, normalization usually doesn't mean something like "like give a unit vector", where they both have value and the normalization is lossy, but rather cleanups of messy stuff that ought not to occurred in the first place.
What I am saying is not the original input has no value, but we should take the steps to make those problems actually not happen in the first place. Rich interfaces do that, and are an investment that pays for itself eventually so this isn't manifestly unreasonable.
> ...metaprogramming over algebraic datatypes not parametric polymorphism...
I worked on a little toy "tree editor" where we did just that. The use of a good type system to both operate "generically" without "upcasting" and loosing the structure for ever is key.
> See also the “nanopass” approach to compilers.
I do like that a lot :) but I also like that there is an IR for every pass, like with CakeML. I thought those went hand in hand?
> (empirically if not theoretically evident) advantage of Unix’s “getta byte” approach to IPC
I would argue the (over-)prevelance of HTTP is effectively empirically the opposite, namely that people do prefer structured things in a vacuum, but coordination failures have repeated gotten us stuck in local maxima of already widely established things like Unix's various binary streams and then HTTP.
> This is why I mentioned Paredit and Dylan’s “skeleton syntax trees”: they are, respectively, a structural editor and a macro system that work where others fail by the virtue of being expressed in terms of more than text or even tokens, but less than a full syntax tree.
I do agree the jump from text to sexprs / json / token trees (Rust's name) etc. is the most important step. Fist get rid of flatness, then worry about higher invariants.
> We’re talking about programming languages, about making a computer understand the programmer’s intent to some degree.
And surely text doesn't help with that. Too much structure makes some edits hard if the user can't temporarily mid-edit ignore the rules, but computer should love it.
> It’s kind of endearing to see the luminaries on the Algol committee (essentially all of CS at the time) state that they want, like, pseudocode, from their papers, except, y’know, with a compiler, and not suspect anything can go wrong with that.
This is about Algol 1968 not Algol 1960? My understanding was that was too many features, not too many invariants.
Agreed. I’m pretty sure I have it solved with Tree Notation though. I’ve got a thing in the works which I call 3D git. Tree languages get it for free. The trick was simplifying the syntax so there’s nothing left but space, and then your syntactic shape matches the semantic shape.
A sneak peak:
https://twitter.com/breckyunits/status/1408546408911695873?s...
The Scala thing isn’t happening, though, — it’s a difference of style and approach, not syntax.
Imagine having an editor which automatically applied your local preferences around tabs/spaces, code formatting, variables up top vs nearest use, function definitions nested to minimize top-scope surface area vs all functions flat at top scope, etc etc etc
And when you are done editing, all of these local changes are reversed and you submit the minimal possible diff.
(and if we want to really talk pipe dreams, the dev only sees an AST editor and the underlying text is never even exposed in the first place)
Our tools are so far behind mostly because everyone’s thinking is still chained to 1970’s hand editing text files mentality. This is the flying car which isn’t being worked on because everyone is still thinking about making better bicycles.
Also - AST isn't THAT important when reading the code. Let's say I give you this:
X (X X X X; X X X; X X) {
X (X X X) {
X(X X X X X)
}
}
Do you know what this code does? How about this: for int i = 0 i < 10 i ++ if i % 2 printf " %d " , i
I've used several graphical languages professionally (not AST-based, graph-based, but the problem remains) and the main problem was - structure wasn't fully describing what happens - the "meat" of the behaviour was still in text form and was hidden behind the pretty graphic form - in case of both of these languages the meat was in the names of the subprocesses called and in the mapping of process variables <-> subprocess parameters.And there were A LOT of these, so you couldn't show them at once on the same screen as the graphical view of the process. So programming with both of these languages was very frustrating - you had to click on each node and look through long lists of x:y substitutions to track how parameters flow through the system.
Also, cities where one can get around on bicyles are much nicer than where one needs to use cars. Flying or otherwise. https://www.youtube.com/watch?v=ul_xzyCDT98
To summarize both points: what looks like progress superficially may actually not be progress at all.
Our tools are so far behind mostly because everyone’s
thinking is still chained to 1970’s hand editing text
files mentality.
People IN THE LITERAL 1970s were talking about this exact idea. We still don't have this, for anything other than highly-specialized applications (for instance, equation editors in word processors).It's almost like it's way, way, way harder to deliver software than it is to handwave about how much better things would be if people just listened to YOU.
Not really the tone I had intended, but it looks like that's the tone which came across.
To be more explicit: "It's a shame there's no money in next-gen tooling"
Another problem is that virtually all of these efforts completely misunderstand what makes programming hard.
git blame, for instance, has the "--ignore-revs" and "--ignore-revs-file" options which let you specify some commits that can be ignored. This is a little helpful for the sort of code spelunking the author does (which I do very frequently), but it's a manual process. There's no built-in convention to Git for what this file should be named and how to have git blame pick it up automatically. Moreover, you have to create the commit in order to know what its revision is, which means that in squash/rebase workflows, you have to land the style cleanup on the target branch and only then can you add it to the ignore list.
One option that occurs to me is a special marker string to put in the commit message like "Git-Ignore-For-Blame: yes" or something. I'm not sure if there's a better way to do it.
More generically, if the commit is only swapping out tabs and spaces, "git blame -w" ought to take care of it. For more involved reformatting, git diff and git merge both have "ignore-all-space" modes, which will handle things like line breaks.
But really what you want for blames and cherry-picks, I think, is blames at the syntax level. The Git commands only know about lines - git blame runs on lines, git cherry-pick constructs diffs based on lines and tries to re-apply them, etc. But at the repository level there's nothing special about line breaks. Why can't we run git blame or cherry-pick on an AST, so that it operates regardless of formatting?
Are there tools that do this? (They don't need to be built into git; a third-party tool ought to be able to do this just fine.)
i bet there are some cool papers out there on trying to do this... but, this is essentially a restatement of the halting problem, so building something that is guaranteed to be correct using only static analysis may be impossible. (but this does not preclude a solution that might be good enough)
has always served me better than blame. It can better jump file boundaries and find e.g. prior code that the code in question was copied from.
I've always thought though that we need better conflict resolution that is code aware. And also better changeset specification, e.g. if a commit is equivalent to a s/foo/bar/ then this information should be included in the commit, or if a function signature has changed, then record that fact, rather than the dumb line by line diffs.
Someone just has to build it.
Reminds me of a project I worked on a while back - whenever you looked at the git history of a file, if you went back far enough you’d hit a dead end at the point where every source file was renamed from *.c to *.cpp.
We have a few files at work that change name with every commit…
Hopefully the next item in this RCS CVS Subversion Git chain is just storing ASTs and transforms on top of them so we can spend less time fixing basic conflicts and discussing formatting.
First, it's too weak - you need to at least recognize special top-level defun-style forms, or you'll generate some minimal diff between two totally different functions just because they both use the same cond pattern or whatever.
Second, reader macros mean you can't really work on the source AST in the first place, unless you also teach the diff tool all your reader macros.
Citation needed
The "problem" is no one actually wants to resolve merges that way.
If you mean Git had issues finding some specific code motion to show in a diff, you can try one of the other diff algorithms, or adjust the threshold for rename/copy detection. AST-based differs would also suffer this issue; a "nice diff" is not a formal problem and does not have a universal solution.
If you mean you once had a mis-merge that dropped some code you didn't want to drop, this won't go away with AST-based diffs. It will just happen at the token level instead of the line level.
If you mean it's generally hard to deal with collapsing lots of branches with shared history, that's true but would also be true with AST-based approaches. This is the situation something like pijul could help with, but also raises all the other tradeoffs of snapshot vs changeset based approaches.
[1]: https://pijul.com/manual/why_pijul.html#comparisons-with-oth...
Never looked at it myself, so not an endorsement. Just came up in the Lobsters discussion[2] on this last week where someone mentioned it.
[1]: https://www.unisonweb.org/docs/tour/
[2]: https://lobste.rs/s/b9pddy/when_it_comes_git_history_less_is...
Your suggestion (what comes after git) is throwing out the baby with the bathwater.
To me the solution is to migrate to trunk based development and hide new features behind feature flags.
The biggest issue with this transformation seems to be implementing feature flags in the code.
Also, is he implying that thousands of commits (one for each file) would be a better way to convert that large code base? Why not solve this problem with the right tools, such as ignorerevs?
It kinda serves as a hook to describe why these changes are bad. I think it's a good example for the rest of the article.
I think the implementers of the Go language got it right: there's one style, for everyone. Like it or hate it, there's no Holy War over it.
Few tools do it out of the box (in the python world flake8 supports diff filtering out of the box, and there's diff-cover which can filter analysis beyond just test coverage) but it's easy enough to do with a wrapper, especially if the linter has a configurable or even parseable (e.g. json) output option.
In this case, the OP is supporting old Linux releases and remarking how naturally cherry picking can work. LLVM by comparison, doesn't have long-term stable releases. So this conservative approach doesn't necessarily apply for LLVM.
So I like the analysis but I disagree with the generality.
Any organization that releases software revisions has this issue. Yes, we do live in a world where much software is now subject to "continuous release" (either applications which are auto-updated to latest, on phones and some laptop OSes or web-only applications where the latest is pushed to deployment). And indeed the specific issue described here is much less important.
However I suspect the majority of software in use is not like that. On the Enterprise side, OSes and applications are updated carefully; an ERP upgrade can be a major undertaking. All sorts of machine control software is ether never upgraded, with limited patching, or treated the same way as other enterprise software. Etc.
The difference between the two is why there is both "git flow" and "GitHub flow" -- it's not like one is absolutely superior to the other.
Then there last paragraph would tell them he’ll ignore replies that don’t mention plaid.