Hacker News Reader

Comment by nailer | Hacker News Reader

Storing changes as text is a fundamentally bad idea. Not only does it make merge conflicts more likely to happen, as transforms (me adding a function, you renaming something I use in the function) more likely to occur, it also means that formatting is committed and discussed rather than being a matter of personal preference.

Hopefully the next item in this RCS CVS Subversion Git chain is just storing ASTs and transforms on top of them so we can spend less time fixing basic conflicts and discussing formatting.

View on HN

Git stores snapshots, not diffs, and therefore could just as well be considered storing ASTs. The trick is writing a useful diff/merge for them. Programmers also don’t think in ASTs, arguably even less so than lines. The problem is not as formal as it looks.

> Programmers also don’t think in ASTs,

Citation needed

T_PAAMAYIM_NEKUDOTAYIM

There’s great irony in someone who’s name is “morelisp” saying that programmers don’t think in ASTs. Lisp syntax is the AST.

Which also means the Lisp AST is not great to diff with.

First, it's too weak - you need to at least recognize special top-level defun-style forms, or you'll generate some minimal diff between two totally different functions just because they both use the same cond pattern or whatever.

Second, reader macros mean you can't really work on the source AST in the first place, unless you also teach the diff tool all your reader macros.

Not sure why you are downvoted. This is a genuinely interesting idea and seems like the basis for next gen change management.

They are being downvoted for not understanding how git works. If you have a 3-way diff/merge tool for ASTs you can plug it into git and use it today, and you can use it on all existing branches and historical changesets.

The "problem" is no one actually wants to resolve merges that way.

There’s been plenty of times my code as been “lost” in a later commit because Git didn’t know it was the same thing.

I don't understand how this relates to what I said without more details about what "lost" means, sorry.

If you mean Git had issues finding some specific code motion to show in a diff, you can try one of the other diff algorithms, or adjust the threshold for rename/copy detection. AST-based differs would also suffer this issue; a "nice diff" is not a formal problem and does not have a universal solution.

If you mean you once had a mis-merge that dropped some code you didn't want to drop, this won't go away with AST-based diffs. It will just happen at the token level instead of the line level.

If you mean it's generally hard to deal with collapsing lots of branches with shared history, that's true but would also be true with AST-based approaches. This is the situation something like pijul could help with, but also raises all the other tradeoffs of snapshot vs changeset based approaches.

You’re still committing formatting, and our whole ecosystem of tools is focused around text. You’re proposing a workaround rather than fixing the problem.

I don’t know, to me whether you store the AST as text or binary sounds like ignoring the problem. Why would that matter?

Because formatting changes aren't relevant.

This isn't specific to VCS, but I'll drop a link to this thread about structural editors which PaniczGodek has been maintaining for a few years now, just to give it more exposure: https://twitter.com/PaniczGodek/status/1195784199250284545

In a sibling comment, I was wondering about projects that are trying ASTs. I haven't read deeply about it, but I recalled that [pijul][1] might be a way forward.

[1]: https://pijul.com/manual/why_pijul.html#comparisons-with-oth...

Pijul's patch algebra, and storing diffs rather than snapshots generally, makes it less amenable to these kinds of experiments than Git. Changing your diff/merge strategy would be akin to rewriting your entire project history.

The way we store data is not the same as using that format to solve conflicts. Take a look at semanticmerge.com

In Tree notation the text and ast have the same shape. You can then have a semantic git

https://arxiv.org/pdf/1703.01192.pdf

I've seen the suggestion of VCS/SCM storing ASTs rather than plaintext for a decade now. Are you aware of any projects that are trying to address this?

I heard Unison[1] does this, or something like it.

Never looked at it myself, so not an endorsement. Just came up in the Lobsters discussion[2] on this last week where someone mentioned it.

[1]: https://www.unisonweb.org/docs/tour/

[2]: https://lobste.rs/s/b9pddy/when_it_comes_git_history_less_is...

Git stores more than just code, though. The tool already supports custom merge tools, so why not apply your favorite AST mergetool to the relevant source files?

Your suggestion (what comes after git) is throwing out the baby with the bathwater.

Does that mean git would have to support all known programming languages, and any language syntax change would require a backwards incompatible upgrade of the VCS?

As well as any known config file formats.

More likely git would support a common AST format/protocol and language tooling would be responsible for providing that.

Yes. Much like we do with editors today.

Reply on news.ycombinator.com