Hacker News Reader

Yes I did an experiment years ago with 6.5 million files and it was disaster (https://breckyunits.com/building-a-treebase-with-6.5-million...).

However, things have totally changed with the M1 generation of Macbooks. Things that were once near impossible now run in an instant. I need to redo this experiment.

View on HN

When I worked at NetApp, this was a problem there too.

IIRC, the fix was for directory entries were later stored in sorted buckets (aka hash map) and only the corresponding bucket for the file name would get locked. This reduced scope of lock for atomic operations like rename and also allowed faster lookup instead of O(n) based scan.

Reply on news.ycombinator.com