This is a fair point. I usually only "jump" from directories that have at most 500k files, and the median is way closer to ~500 files.

Even in the worst cases, at least with my i7 NUC and using fd instead of find, it's less than 500ms to load the entire list into fzf.