Highlights from Git 2.34
github.blog
github.blog
I have one question: how do changes to files in the working copy automatically update the current commit hash? Do you just calculate it on demand, and then actually make the commit when some command would move to different commit (`jj switch someotherbranch` or whatever)?
In case you're thinking that that seems slow, it's not really significant at least up to git.git scale. For larger repos, I plan to rely on something like watchman.
One issue with two-way interop is that while importing a commit into a patch is easy, doing the opposite is more complicated. Indeed, since Pijul is patch-based, there is no way to map patches onto commits in a unique way, since a single patch is quite independent from the context (other patches around it), which is what makes Pijul easy to use.
Also, if Alice and Bob work on a common Pijul server in parallel with their Git server, they'll get artificial conflicts in Pijul, even though they have the same contents.
At the moment, Pijul is in a state where there is just one remaining feature I want to implement before beta, with a massive expected impact on disk space (and a limited impact on speed as well).
For the kind of "smaller (but essential) features" like what you're suggestion, there are tons of those, and we can totally implement them after beta. Git export is a common request we get, though. I'm just scared of how people will react when they export and then reimport their patches elsewhere, and get tons of fake conflicts.
Regardless though, pijul is really cool. I just badly want to be able to use it at my day job :)
I wonder if it wouldn't be possible to memorize "git order" of patches when importing them in Pijul to be able to re-export them in the same order as they were imported before.
Are there any specific things that git does that you wish were done better? I probably won't be able to help with very generic things like "it's complex", but if there's specific drawbacks in some particular commands I might be able to fix them.
Or anything else, I'll monitor this thread while there's interest.
If I want to quit/escape adding a patch in the interactive mode I type `q` and then enter.
If I want to quit/escape the interactive add subcommand I type `q` and then enter.
But if I want to quit the `update`, `untracked` etc. I have to enter nothing and then press enter. It would be such a wonderful thing if it also accepted `q` so my muscle "memory" is correct.
I looked into changing this once, but it is written in shell and Perl, which I don't know enough about to figure out.
It looks like the ball may have been dropped on opting users in to it by default, but in any case there's numerous bugs in "git add -p" and "git add -i" that got fixed along the way, and fixing existing ones should be a lot easier now.
Note: I've also worked on the perl compiler/interpreter in a past life, so it's not that I'm afraid of hacking on Perl, but doing so in the context of the Git project is and was always particularly painful. If something's written in C you could use all the APIs git had internally, but anything in .sh or .perl was usually stuck parsing text, or reinventing the wheel.
The other thing that was frustrating with that was that the help text for what commands do is printed before everything, so if your window is small and there's a lot of output, it's not obvious that the ? you typed did anything other than re-print the hunk. It seems odd to not print the help information at the end so you will always see it (unless your window is less than one row tall, but.. you're probably here for dragons anyway in that situation).
For example when I try to bring an old feature branch forward, or when I have two private branches with a common history that I've touched on one side and want rebase to apply my history change onto the other branch, I'm often made to resolve a lot more conflict that seems necessary.
There are scripts like git-imerge that give some much nicer conflicts. imerge may not be perfect, but I think it shows there's room for a more clever algorithm.
I'd happily have a git rebase --full-resolver --wuauctl option that consumes a mysteriously large amounts of CPU time, if it saves me some manual conflict resolution =)
Unless that UX problem is helped by it being much, much faster, but that's usually only going to be matter for larger repositories.
[alias]
permission-reset = "!git diff -p | grep -E \"^(diff|old mode|new mode)\" | sed -e \"s/^old/NEW/;s/^new/old/;s/^NEW/new/\" | git apply"
I wish `git-log`'s `--follow` wasn't optional so GitHub et al would show the complete history for files I `git mv`. Mercurial got this one right :)
I'd say that most of the work of driving features forward in git is hunting down those various edge cases, taking the time to understand them, and explaining them to others. So if anyone's interested in flipping that particular switch it's very open to first-time contributions.
I know file locks are almost an antipattern for git. But there are definitely good use cases and arguments for better git support and related tooling support around them, especially for binary heavier areas.
The other thing I saw other day was sparse index or something. At work we have a monorepo and I wish that was more control or granularity over checking out a subset of a repo somehow. Our clones even with shallow fetch and lfs can be 5+ minutes for an area that should only take 3 seconds. But no good subset checkout.
1. A consistent cross-language API for reading and writing from a bare/remote git repo in ways that would make sense to an average programmer. Something like `git.stat("/file.txt", commit=...)`, `git.open("/file.txt", branch=...)` `git.list_commits(since=...)`, `git.create_commit(parrent_commits=[], branch="", ...)`
2. A good story, better than LFS, for storing large binaries over time.
For #2 I want to reproducibly build a debian image from vendored packages or recompile gcc from source so I need to check in a pre-built compiler. etc
I really like the idea, and wrote my personal little `git autosquash` command that tries to figure out the oldest commit it has to run `git rebase -ir --autosquash` on in order to merge all current "fixup! …" commits into their counterparts.
However, doing this in a simple shell script is rather dodgy if you run into more generic commit messages. Personally, I'd love it if git allowed me to create "fixup! <commit hash>" commits instead. I mean -- it's a fixup. It's not supposed to be readable anyway.
Do you mean that you've got commit subject that are ambiguous within the series you're rebasing, so --autosquash won't correctly attribute them?
What I want to achieve is that I can run my script (i.e. `git autosquash`), and it will automatically start the appropriate interactive rebase for me. That means I still get the rebase description I can check by eye in Vim (and have the option of aborting by emptying the buffer before quitting it).
I usually use this to clean up the history of a private local branch, so I won't trigger anything like `git rebase master`. This means that I have to specify the commit where Git is supposed to start the interactive rebase (i.e. the oldest commit that won't be included). And finding that by looking for the subject line after "fixup!" is rather finnicky (especially if the subject line also matches those of other commits), whereas the simple commit hash would be nice and explicit (and finding the starting point would be a simple `git merge-base --octopus` of all fixup! hashes).
So, my problem isn't the autosquash resolution built into Git, but that I have to reimplement it in a script in order to find the proper rebase starting point. That, and all backward searches for commit messages I know of are either really fuzzy or needlessly convoluted.
The best exact matching search I could come up with is something akin to
git log --pretty=$'%H\x1F%s' "$fixup_commit_hash" | awk -F $'\x1F' -v "subject=$query" '$2 == subject { print $1; exit }'Is there any reason you don't (or perhaps you're not aware that you can) use the branch upstream info for this?
Whenever I work on a branch called "topic" I do the equivalent of "git branch --set-upstream-to origin/master" on it. That means that all of:
git log @{u}..
git diff @{u}
git diff @{u}...
git rebase -i
git rebase -i @{u}
A foo
B bar
C !fixup bar
D !fixup bar
> You'd open an editor with A..D with that, whereas you perhaps only want B..D?
That wouldn't be much of an issue for me, but I still tried to implement an option that doesn't rely on my repository having a defined remote.
I quite often start a new local git repository whenever I'm about to start a new experiment (or just scripting some one-off task), and most of these will never get pushed onto a git server. However, even in those cases, my scatterbrain often triggers the "gah, that should have been in a previous commit!" reflex.
By the way: Thank you very much for taking your time and reading all this! :)
Edit: I just re-read the upstream-point thing. Wouldn't that cause a `git push` on `thing` to push directly to `origin/master`?
That also wouldn't be compatible with the workflow we have at work (we use GitLab merge requests)…
I use this extensively with Git(Hub|Lab) (PR|MR)'s. I set the upstream to "master" but push to "topic". The "push.default" docs cover how to have your cake and eat it too there.
When auditing a source repository you want to see how commits are signed over time. In other words you have a key X which is valid at time Y. As Y changes, key X will change as well. To audit a repo, you need to know the history of what key X was valid at time Y.
With GPG you can set up a hierarchy of commit signing keys which have defined lifetimes and are all subordinate to a master key which you do not use for signing (and hopefully keep somewhere secure). One irritant is that the major git-as-a-service products, i.e. github + gitlab, do not respect the use of GPG keys as something varying over time or subordinate to a master key. Instead they let you set a current active key and mark on their own if it was verified at the time. If you want to validate the repository's security yourself, you don't have the proper information to do so.
The use of ssh keys as signing keys furthers this incorrect usage of a single key as something that does not vary over time. You can use ssh certificates as a way to have a hierarchy, but that still doesn't acknowledge that there is historical data of note.
Now, a logical question is: If each git commit is the summation of everything before, why does historical data matter? Why can't I just trust the git{hub,lab} when it says it was "Verified" at the time? A few reasons:
- Github allows someone logged in to make edits via their web editor which they helpfully sign with a universal github key that is always verified. So if your AuthN is bad, that is a problem.
- You are trusting that your git-as-a-service provider will never have their repositories tampered with and your org will just download the newest version of the repo.
- It relies on git-as-a-service providers to track this. If you aren't using a managed service provider you need to roll your own auditing tools.
I understand your reasoning and the idea behind GPG and everything, but here's my challenge to all of this: Has this ever happened?
Like, the scenario we're talking about is that at some point some bad code, maybe a backdoor, got introduced into a code repo, and it later is discovered and someone launches an investigation. And they find out "this was committed by XYZ but XYZ claims he didn't do it".
I don't remember any such incident ever. Given that I feel a slightly weaker form with SSH keys may still be justified. (And it is still easily conceivable to have some out-of-band-metadata, e.g. a project that feels this is important could just have a list of developer SSH key fingerprints within the repo and thus have the backlog of who's keys were relevant at what point in time.)
Unfortunately this has: https://www.wired.com/2015/12/researchers-solve-the-juniper-...
While the full results were not released, it was pretty clear that a rogue commit happened to the VCS. This is the highest profile example that has occurred in my corner of the tech world.
The broader category of this attack is "insider threat" and it is a real thing. Even googling for "malicious code commit" turns up this other example for me as one of the first results: https://www.theregister.com/2021/03/29/php_repository_infect...
Notably, I'm not just asking if there ever was a bad commit that was investigated - but that the commit came from someone who later claimed she/he didn't do it.
> With GPG you can set up a hierarchy of commit signing keys which have defined lifetimes and are all subordinate to a master key which you do not use for signing (and hopefully keep somewhere secure).
Within epsilon of nobody is going to do this, but they all have an SSH key. Heck, I don't do this, and I even actually know how to make GPG do it. It's a huge hassle for what, at this point, is really only useful to me for...signing Git commits? I don't need to sign anything else with it, and GPG mail is nonexistent.
(Also, fwiw, if you aren't using a managed service provider, I hope you have a directory service where you can have those keys. If you aren't using a managed service provider and you don't have a directory service, I'm gonna go out on a limb and say that I'm guessing security isn't that important to you.)
https://man.openbsd.org/ssh-keygen#ALLOWED_SIGNERS
When validating the SSH signature, Git can set the verify-time option to the commit timestamp and the signature will be rejected if the key is not valid at that time. Git doesn't yet do this, but will in a future version.
This is considerably simpler than the rigamarole of GPG subkeys.
Being able to sign commits with your SSH keys makes signing actually useful, because it enables a new workflow that developers will use:
- You give every dev on your team a Yubikey
- They generate an ed25519-sk key that only resides on the Yubikey, no software required as it works out of the box with both openssh and GitHub
- They upload the public ID of the key to GitHub, same as before
- You enforce commit signature verification for your GitHub org. You're done, no need to install any software, everything Just Works.
You now have:
- No private keys on developers machines, rendering all types of supply chain attacks like NPM stealing your .ssh files ineffective
- Enforced 2FA for everyone without any hassle
- Every commit signed by developers, enforced and with no developer overhead. Checks a lot of boxes for those SOCs and ISOs.
How does this work? Can you like me a ressource?
For more details, see:
https://github.blog/2021-05-10-security-keys-supported-ssh-g...
https://www.yubico.com/blog/github-now-supports-ssh-security...
Would it be able to get the key from the usb device? Or does the hashing happen on the device itself with the private key never exposed? If that's how it works, that would be awesome, but would a malicious program be able to sign something as you, instead of stealing the keys?
I'm going to look into this more and maybe recommend it at work.
If you could extract the key, you probably have access to the intelligence resources of a very large nation. With hardware security dongles, the key remains on the dongle; you pass your data in and it passes the signature (or encrypted data) back out.
> In Git 2.34, ort is now the default merge strategy, so you should notice faster merges with fewer bugs just by upgrading
> a rewrite allowed Git to implement a merge strategy that doesn’t operate on the index
Merge of two good commits will silently produce bad commit.
Merges are not repeatable and can't be reasoned about.
Merges get in a way of git bisect.
Merge makes simple "git log" not contain all the log.
Good luck cherry-picking a merge commit.
Merge makes a commit which has important changes not apply to any specific revision of git log.
It's an enormous foot gun and a single contributor may inadvertenly taint your repository for life.
Merge is a make-believe working set operation, like if you could pretend to add apple to mango and get a pomelo.
Merge is VCS analog of lead acetate. Something you have no reason to ever put in your mouth.
git-2.34.0.tar.gz 15-Nov-2021 16:48 10M
git-2.34.0.tar.sign 15-Nov-2021 16:48 566
git-2.34.0.tar.xz 15-Nov-2021 16:48 6Mbtw rhel maintains a 1.8.x release.
Possibly; I've been hearing disturbing things about developments like different hash algorithms, and repositories being generated that older versions of git cannot read.
I'm nervous about creating a new central repository with newer git.
It works fine in that I'm not running into issues and not missing features, not in the sense that I proved that it's free of defects; I don't require a fork that has any new commits. Surely, bugs have been found between that version and now, some of which can be meaningfully backported.
Programs can break simply because of all the other moving parts in the environments that are out there; it's a fact of life nowadays that software needs to have a maintenance plan to be viable. A compiler, library or kernel upgrade can expose some latent problem in a program.
Continuing Git's famously clear naming I see!
> When COMMIT_ID_DEFINE_LOCAL_CACHE is not patched, the most recent counted grafts chased by submodules in a subtree, but that are not in other-bundle, may be chased by an active head ref. Using sub-recursed or pre-massaged flags only cautiously grips the tag, except when otherwise noted.
Comments from the last time: https://news.ycombinator.com/item?id=28207168
For those who don’t know: GitHub is owned by Microsoft, a historical (and current) adversary of all things not Microsoft, particularly Linux¹. Git, the tool, is not made by GitHub; GitHub is a web service which is using Git. Git, the tool, was originally made by the person who also originally made (and currently manages) Linux, and the purpose of Git was to manage the development of Linux itself.
I think that most people are already under the misapprehension that Git and GitHub are the same thing. This style of deliberate confusion from GitHub must be seen to be by design. They are Microsoft; they do not get the benefit of the doubt.
It's pretty clear they're talking about a separate project.
Really? This seems just very much an anti Microsoft view, especially with that final sentence.
Yeah actually, a lot are. There are also a lot of GitHub users who don't know what git is at all.
I don't think this blog post is going to further misinform them though, they also don't read technical blog posts.
GitHub employs at least three full time git contributors, including the author of this blog post.
(Disclaimer: I am a GitHub employee who works on git-adjacent things)
Um - I thought Microsoft does have a big role - doesn't Johannes do the Git for Windows stuff and work at Microsoft, and Jeff work for github etc etc.
It's perfectly reasonable for a company, that uses the tool in its name, to have a blog post going over new features. I'm never amazed the distances people will go to distort a boring blog post into some issue or another based on splitting hairs.
Off topic, but there was that dotnetfederation overreach shenanigans
For the record, I actually did give Edge Chromium a spin and I was a bit impressed. The Deals feature creeps me out, but I can browse the reddit homepage without an ad blocker and it's actually a smooth experience.
Edge is probably nice, forcing users to use it is not.
One should think they'd learned after being forced to advertise other browsers for a couple of years?
[edit: in fact it's linked from GitHub's blog post, at the bottom, and it's included in the announcement email: https://lore.kernel.org/git/xmqq8rxpgwki.fsf@gitster.g/]