Hacker News Reader

> I would maybe be interested in Git allowing you to plug in your own diff generators for different file types.

This is already supported.

A common example is UnityYAMLMerge for merging the Unity game engine's generated files.

https://docs.unity3d.com/Manual/SmartMerge.html

Configuring it to work with Git and others is a little ways down the page, but would apply the same for other diff tools.

I looked this up and for anyone wondering, it's called "diff/merge drivers", but there are only a handful of them out there. Some highlights from a few minutes of searching:

- MS Office: https://github.com/lcnittl/DMFO - SQLite: https://github.com/cannadayr/git-sqlite - Jupyter notebooks: https://nbdime.readthedocs.io/en/latest/vcs.html#git-integra...

One big caveat of this is that since git doesn't really store just a stack of diffs, despite the fact it presents itself as such to the user, a custom merge driver will not make your .git grow any less than it would normally.

> One big caveat of this is that since git doesn't really store just a stack of diffs, despite the fact it presents itself as such to the user, a custom merge driver will not make your .git grow any less than it would normally.

Note that git does support using deltas for storage. But according to docs, custom diff drivers aren't used for those, instead it's a instruction based format.

https://git-scm.com/docs/pack-format#_deltified_representati...

>One big caveat of this is that since git doesn't really store just a stack of diffs, despite the fact it presents itself as such to the user,

>a custom merge driver will not make your .git grow any less than it would normally.

You can go a long way with simple homespun methods. SQL dumps delta-compress in Git very well - especially if you tweak the dump options a bit so that the row order is mostly stable.

The same most probably goes for any other textual dump - if the (unchanged) content is largely ordered in stable fashion, it will delta-compress very well, even without any explicit, specialized support in Git itself.

I investigated the option of using a custom git merge driver for a project where we were planning to version control a bunch of data files using git.

Here's a proof of concept python merge driver I bashed together at the time to auto-merge JSON objects: https://github.com/fcostin/jsonmerge_git_merge_driver

This never went anywhere near production, but it was very easy to put together something basic.

One complication with using a custom merge driver, as discussed by https://github.com/Praqma/git-merge-driver , is that they need to be configured inside the `.git/config` of the repo, which itself is not version controlled. So there's an additional config management overhead to rolling that out to everyone in a machine. Additionally, if outsourcing hosting for git repos, it may not be supported to install and configure a custom merge driver for merges conducted by the hosting platform (e.g. merges created by github.com pull request workflow).

One idea I had at the time was using external schema files (e.g. JSON schema for JSON files) to help guide/constrain the result of the merge. I never implemented it, but it should be possible. If the schemas were also version controlled in the same git repo that stores the data, you'd need to figure out which one (and which version) to load when resolving a merge conflict of a data file. There doesn't seem to be a well-supported robust way for a merge driver script to discover the source and destination branches, but there are some potentially fragile ways of doing it that work some of the time.

Next you will be telling me that git is already a blockchain!

> Not every programming language's grammar maps to an abstract syntax tree

Are there some examples of this?

Perl, this is because you can't actually properly parse Perl without also running Perl code at the same time.

https://metacpan.org/pod/PPI

PPI and PPR will parse most perl programs, but the moment you start doing stuff with prototypes or source filters they fail completely.

That you can't properly parse Perl without also running Perl doesn't mean it doesn't have an AST. I don't know about Perl so it could very well be that it does not have an AST but your argument doesn't show it does not.

Perl code changes how the parsing will be done, both by adding keywords, changing how many arguments the parser will look for, and even arbitrarily changing the language entirely with a source filter. It's largely that last bit that makes creating a consistent or coherent AST so impractical it may as well be impossible.

The AST contains code, but it's still an AST. Every AST contains code. Perl has a dynamic lexer, but still uses a static YACC grammar.

Forth. You could certainly define a formal grammar for it and construct an AST, but it would be trivial and not very useful.

I disagree. Many engineers want to refactor across a sequence of small PRs, for example. Small PRs are a good thing, because they’re easier to understand. But today, Git makes this painful. Also, understanding how the meaning of code changes over time can help reduce bugs.

The solution will have to be pluggable. But I think it is possible, and there are sane things to do (e.g. fall back to vanilla git) when there are missing plugs.

Not only that, but imagine you realize there is a bug in the parsing tool. Now you have to go back and re-parse the code, or otherwise just deal with a bad history forever. Suddenly you’re storing text again.