Git v2.20.0 | Hacker News Reader

Git v2.20.0

lkml.iu.edu

93 pointsjjuhl16 comments

This release incorporates "delta islands" which is code GitHub has been using since 2015-ish (or earlier?) to store repositories and their forks in the same object storage, with "islands" of objects as they diverge. See https://githubengineering.com/counting-objects/ for a related blogpost of theirs.

This was work sponsored by GitLab, and I know they plan to use it in production sometime soon. So it's interesting for other similar hosting sites with a "fork" feature to look into it.

Your post as of my writing this comment makes it unclear: Is the code in this specific release written by GitHub, GitLab, or both?

If I had to guess:

Github added the feature to their platform, but it didn't go upstream. Gitlab sponsored the dev-hours to rewrite the feature and actually push it upstream

It was written by GitHub, and has been used in their internal fork of git.git for a long time. The work of integrating it with upstream was sponsored by GitLab, since it's something they were interested in to save space.

The feature isn't interesting except for those that run a GitHub-like setup. I.e. repositories with a fork network where clones & fetches can be expected to be served from any repository in the network. In those setups it can make a lot of difference.

With gitlab-ce being ~500mb this must be a gigantic improvement for them.

> It was written by GitHub, and has been used in their internal fork of git.git for a long time. The work of integrating it with upstream was sponsored by GitLab, since it's something they were interested in to save space.

That makes it clear, thank you. My assumption was that GitLab was copying work done by GitHub.

> The feature isn't interesting except for those that run a GitHub-like setup.

Sure, doesn't mean I'm not interested.

If the last chart in that post[1] is anything to go by, then we will hopefully see some significant performance improvements from this.

[1]: https://githubengineering.com/counting-objects/#the-results

When do we get large binary support? (I know there are external projects for this, but they are not nearly as convenient as they could be, as you'll have to run separate server daemons, do separate configuration, etc.)

How would you implement large file support without running a daemon to host the files?

Using the same daemon as git uses itself. E.g. ssh.

This is slowly but surely being worked on. For one, Microsoft has been upstreaming parts of their GVFS fork, if you search for "partial clone" in release notes since 2.16.0 (including in this one) you'll see some of that work relevant to your question. I.e. one part of this is the general ability to have partially available local history, whether it's skipping (big) blobs, some trees etc.

Another effort has been the introduction of the v2 protocol to Git, which has happened recently, and is only now starting to get rolled out at various hosting providers. That in and of itself hasn't helped with this, but allows for future extensions to the protocol, such as "this is not the full data, you can find the rest at xyz".

Then there's the "odb" effort, see e.g. here: https://public-inbox.org/git/20180802061505.2983-1-chriscool...

As to your "when" question. That's not really a question anyone has the answer to. It's hard to change a piece of software used by millions when you need forward & backwards compatibility, a lot harder than e.g. one company (such as Microsoft) rolling out similar features for their internal users.

But a lot of people are interested in it, and steady progress is being made towards it. It's not an area I work on (I hack on other things in git.git), but I think the above summary is somewhat fair, and hopefully it helps a bit.

Note that while "large binary support" doesn't need to be synonymous with "partially downloaded history", in practice that's what most people mean and want from such a feature. So I've assumed that that's the question you asked.

Is there much stopping pack files being compressed with something else besides zlib?

I install git by compiling from source, as Ubuntu's version is behind a bit.

My normal set of options isn't working.

./configure --with-openssl --with-libpcre2 --with-curl --with-expat --with-pager=less --with-editor=vim --with-tcltk

    LINK git-credential-store
    LINK git-fast-import
    LINK git-daemon
    LINK git-imap-send
    LINK git-http-backend
    LINK git-sh-i18n--envsubst
    LINK git-shell
    LINK git-remote-testsvn

imap-send.o: In function `sk_GENERAL_NAME_num': /usr/include/openssl/x509v3.h:165: undefined reference to `OPENSSL_sk_num' imap-send.o: In function `sk_GENERAL_NAME_value': /usr/include/openssl/x509v3.h:165: undefined reference to `OPENSSL_sk_value' imap-send.o: In function `sk_GENERAL_NAME_pop_free': /usr/include/openssl/x509v3.h:165: undefined reference to `OPENSSL_sk_pop_free' /usr/include/openssl/x509v3.h:165: undefined reference to `OPENSSL_sk_pop_free' imap-send.o: In function `ssl_socket_connect': /home/aaron/Downloads/git-2.20.0/imap-send.c:287: undefined reference to `OPENSSL_init_ssl' /home/aaron/Downloads/git-2.20.0/imap-send.c:288: undefined reference to `OPENSSL_init_ssl' /home/aaron/Downloads/git-2.20.0/imap-send.c:290: undefined reference to `TLS_method' /home/aaron/Downloads/git-2.20.0/imap-send.c:303: undefined reference to `SSL_CTX_set_options' collect2: error: ld returned 1 exit status Makefile:2374: recipe for target 'git-imap-send' failed make: * [git-imap-send] Error 1

You're probably having problems with your openssl installation. Perhaps ask on a place like stackoverflow, where the presentation format can be a lot more pleasing.

Sorry downvoters. This wasn't a good idea. I didn't think of SO.

You can use this PPA to get “bleeding edge” versions of git: https://launchpad.net/~git-core/+archive/ubuntu/ppa

Reply on news.ycombinator.com