Hacker News Reader

Comment by nickcw | Hacker News Reader

It is interesting that the speed of lstat on macOS is the driver behind this problem. According to the article it is 10x slower than Linux.

I wonder if anyone has tried attacking that end of the problem? Faster lstat on macOS would benefit all applications not just git.

View on HN

https://gregoryszorc.com/blog/2018/10/29/global-kernel-locks... is an interesting write up about the problem.

That is also what I immediately noticed. The graphs showed that the slow runs were around ~4s (p50) or ~7s (p90), and then also mention that Linux was 5-10x faster. This would mean the same operations on Linux are somewhere around 1s in the worst case, which is perhaps even faster than their sped up system.

But instead they had to build all this extra junk like an additional caching layer and daemon (wasting memory and CPU) atop the kernel's existing cache just to work around the kernel being slow.

One thing for Linux is that it happily caches large numbers of inodes, as long as there's enough memory for it. MacOS has a (configurable) maximum, and the default value is low enough that it makes things really bad for large repositories. It used to be a few thousands, it's now around 263K, apparently. Check it out on your machines:

  sysctl kern.maxvnodes

> I wonder if anyone has tried attacking that end of the problem? Faster lstat on macOS would benefit all applications not just git.

One major blocker is that the code for this is largely closed source and not accepting patches.

I believe the same thing is a problem on Windows, too. Linux has fast lstat.

Reply on news.ycombinator.com