Are there any other git features with this limitation? Wild to me that we're here.
Thankfully the article covers the semi-longstanding "hooks" that existing (& very high performance) tools like Watchman (which are cross platform) can use.
Great in depth read. Good stuff! From the 2.37 release[1].
[1] https://github.blog/2022-06-27-highlights-from-git-2-37/ https://news.ycombinator.com/item?id=31898261 (34 points, 2 days ago, 7 comments)
Apparently an older implementation using inotify was dropped because inotify does not work recursively, so you would have to do an inotify call for all directories of the hierarchy which is obviously very inefficient. There are system wide limits in the number of directories you can listen to, and even if you increase the limit you would probably cause a lot of overhead.
Newer linuxes support the fanotify system call, which does allow recursive listening. They haven't implemented something using fanotify yet however.
https://lore.kernel.org/git/e1442a04-7c68-0a7a-6e95-304854ad...
https://man7.org/linux/man-pages/man7/fanotify.7.html
In the original fanotify API,
only a limited set of events was supported. In particular, there
was no support for create, delete, and move events. The support
for those events was added in Linux 5.1. (See inotify(7) for
details of an API that did notify those events pre Linux 5.1.)
And https://lore.kernel.org/git/87czs1d6uy.fsf@evledraar.gmail.c... to link to fanotify discussion on the git mailing list (as well as some flaming about the feature not being supported on linux).
inotify has a number of other relevant limitations, like not being able to create recursive notifications or handle "move" operations. Implementation effort is going to be way higher for an inotify-based system, and of course that's made far worse by the numerous file systems in linux - I imagine any implementation would probably start first with ext4.
I suspect an ideal solution would be via ebpf, but I'm not sure.
Of course, in a large enough monorepo Linux performance would also suffer, but to a much lesser degree.
Also, conveniently, both Windows and macOS have an API for recursive directory watch, whereas Linux doesn't (in Vanilla kernel). Inotify can only watch the immediate directory you're observing + there's a pretty low default limit on the number of inotify descriptors that you're allowed to have on top of that
$ time git status
real 0m0.324s
user 0m0.197s
sys 0m0.425s