Hacker News Reader

Oh I have a repository of tens of millions of small json files, and xfs chokes on that specific partition it's amazing. 5 minutes for a ls in the root folder (with only 2 folders). Tried moving it disk to disk in case I was on a broken disk, but no.

View on HN

Unless there is a good reason, I think in that situation I would probably try to avoid storing those tens of millions of small json files directly on the filesystem. For example : have you considered storing that data in jsonb/postgresql https://www.compose.com/articles/faster-operations-with-the-... ? (or even just in an sqlite db https://www.sqlite.org/fasterthanfs.html )

Reply on news.ycombinator.com