I have thought about this a bit in the past, and I think the answer is that the AST for a given blob of text can be different depending on which version of the language you interpret it with. consider 'print "foo"' in python 2.7:
Python 2.7.15 (default, Feb 14 2019, 16:26:56)
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.10.44.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.dump(ast.parse('print "foo"'))
"Module(body=[Print(dest=None, values=[Str(s='foo')], nl=True)])"
but in python 3.7:
Python 3.7.2 (default, Feb 14 2019, 16:12:42)
[Clang 10.0.0 (clang-1000.10.44.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.parse('print "foo"')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/philsnow/.pyenv/versions/3.7.2/lib/python3.7/ast.py", line 35, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 1
print "foo"
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("foo")?
nothing changed in the text, but the AST is different. What should get committed to this hypothetical "git-ast"?
The answer probably lies in creating a different abstraction , different building blocks. In git we have bytes -> blobs -> trees -> commits, where the bytes of a given file get annotated with a file name (and mode bits etc etc, gross posix things) by the tree object.
In git-ast, maybe the abstraction is bytes -> lexemes (which annotates the bytes with a language version, and thus lexer version) -> tree? I haven't gone down the thought experiment that far.