wetneb

In Mergiraf, as soon as there is a parsing error in any of the revisions, it falls back on line-based merging, even though tree-sitter is generally good at isolating the error. It felt like the safest thing to do (maybe we detected the language wrong), but I'm definitely open to reconsidering…

Yeah at the moment it just supports whatever the tree-sitter parser accepts, period. A bring-your-own-grammar version could be interesting, I don't see why it couldn't work. Do you have any Rust crates to recommend, to do parsing according to a grammar supplied by the user at run time? It's likely to be slower, but maybe not prohibitively so…

Another approach would be for the tool to accept doing structured merging even if there are error nodes in the parsed tree. If those error span the parts of the file where the extended language is used, then the tool could still help with merging the other parts, treating the errors as atomic blocks. I'd be a bit reluctant to do that, because there could be errors for all sorts of other reasons.

Thanks for the details. Concerning matching for diffing vs for merging, the differences I can think of are:

- for diffing, the matching of the leaves is what matters the most, for merging the internal nodes are more important,

- for diffing, it feels more acceptable to restrict the matching to be monotonous on the leaves since it's difficult to visually represent moves if you can detect them. For merging, supporting moves is more interesting as it lets you replay changes on the moved element,

- diffing needs to be faster than merging, so the accuracy/speed tradeoffs can be different.

Packaging parsers into separate executables seems like hard work indeed! I assume you also considered fixing the tree-sitter grammars (vendoring them as needed, if the fixes can't be upstreamed)? Tree-sitter parsers are being used for a lot more than syntax highlighting these days (for instance GitHub's "Symbols" panel) so I would imagine maintainers should be open to making grammars more faithful to the official specs. I'm not particularly looking forward to maintaining dozens of forked grammars but it still feels a lot easier than writing parsers in different languages. I guess you have different distribution constraints also.

Lombok is an interesting example, but yes, just with reflection you can already get order-dependent behaviors as the docs note. I've been thinking about giving users more control over this commutativity, but it's not clear to me what it should look like. A strict mode where commutativity is disabled entirely? The ability to disable certain commutative parents?

Thanks for the insightful comments! You surely have a lot more experience than me there, but my impression was that producing visual diffs and merging files are tasks that put different requirements on the tree matching algorithms, and Dijkstra-style approaches felt more fitting for diffs than for merging, so that's why I went for GumTree as it seemed to be the state of the art for merging. Does SemanticDiff offer a merge driver? I could only find documentation about diffing on the website.

As to mismatches: yes, they are bound to happen in some cases. Even for line-based diffing, Git uses rather convoluted heuristics to avoid them (with the "histogram" diff algorithm), but they can't be completely ruled out there either. I hope that with enough safeguards (helper to review merges, downstream consistency checks with local fall-back to line-based diffing) they can be lived with. I'm happy to try other matching algorithms if they are more promising though (there isn't much coupling with the rest of the pipeline).

Concerning tree-sitter, I have noticed some small issues, but nothing that was a show-stopper so far. I actually like it that it's designed for syntax highlighting, because it's really helpful that the representations it gives stay faithful to the original source, to avoid introducing reformatting noise in the merging process. Parsers written for a specific language can sometimes be too zealous (stripping comments out, doing some normalizations behind your back). That's a problem in Spork (which uses Spoon, a pretty advanced Java parser). And the uniform API tree-sitter offers over all those parsers is just too good to give up, in my opinion.

It's based on tree-sitter indeed: https://mergiraf.org/adding-a-language.html

Out of curiosity, which languages would you be interested in?

Yes, see this example: https://mergiraf.org/conflicts.html#line-based-merges

There are attempts to do that, such as https://arxiv.org/abs/2111.11904 or https://arxiv.org/abs/2109.00084. I couldn't find any open source implementation though. I'm personally not so keen to use LLMs for merging: I want it to be quick and predictable.

I tried your example but git does create a conflict in my case - but maybe I misunderstood the scenario. Python support can likely be done (I would be thrilled if someone made a PR for it), but I don't know if there is a lot of potential for solving conflicts there: imports can have side effects, function arguments are complicated with the mixture of positional and keyword arguments, decorators are effectful… it seems to me that there is a lot of sensitivity to order in many places.

It's definitely something I would recommend in general, but I'm not sure if it would solve this particular problem (reordering blocks is perhaps a bit bold for a prettifier).

For now, you let it reorder every child within a given node type, which felt expressive enough to me in most cases, but I agree it would be good to refine that: https://codeberg.org/mergiraf/mergiraf/issues/6

Yes, that's definitely something that could be refined, for instance by specifying that only children of specific types can be reordered together: https://codeberg.org/mergiraf/mergiraf/issues/6