Hacker News Reader

Comment by didgetmaster | Hacker News Reader

Can you imagine trying to have meaningful file names today on these new 20+ TB drives that hold a few hundred million files, if file names were still limited to 8 or even 14 characters?

View on HN

Yeh, but everyone still has DOS PTSD. Underscores instead of spaces, sticking to 7bit ascii characters, etc. so not much has changed. On the rare occasion that I need a slash in a filename, I've been using the full-width solidus unicode character...

Also discovered that I could fool the Macos Finder file sorting the other day if I put zero-width spaces in between numerals in the filename. Not sure how I feel about that one though.

> Underscores instead of spaces

After being biten so many times with spaces or special characters in filenames, one learns.

PATH_MAX is still comically small today.

> PATH_MAX is still comically small today.

Per getconf -a, I see PATH_MAX (and _POSIX_PATH_MAX) as 4096. Is that small? What would be not-small?

Yes that's small, it's also incorrect. The correct thing is to recognize PATH_MAX can't be defined by the kernel or in limits.h, so what you're seeing is a hint and not the actual limit.

"Not small" is "limited only by resource constraints." Software often breaks when it hits large (but correct!) paths even though there's no technical limitation to using them, and valid ways to construct them, even if POSIX APIs are required by spec to fail for some valid paths because of arbitrary limits.

Linux is actually pretty good about ignoring unnecessary error conditions even if it violates the spec, other unixes not so much.

The Linux kernel doesn't actually support path names longer than 4095 (4096 with NUL). See fs/namei.c:getname_flags, which is used early in every syscall that takes a file path. That function will attempt to copy the path into kernel space, but fails when the length exceeds 4095: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... (strncpy_from_user is defined in lib/strncpy_from_user.c)

IIRC, some real-world systems set PATH_MAX to INT_MAX (Solaris?), but I don't know if any modern Unix systems support arbitrary length paths. Everybody demanded features that required the kernel to cache the path in the kernel, at which point the notion of just walking the userspace path buffer (no separate allocation, no need for a limit) went out the window.

EDIT: glibc sets NL_TEXTMAX to INT_MAX. I always get confused when discussing this issue. NL_TEXTMAX made good on the threat that these MAX macros might be (effectively) unlimited and so shouldn't be used to size buffers, but I don't think any system ever did so with PATH_MAX.

So, a user should be able to exhaust resources by making paths longer and longer until it just errors out?

And programmers should allocate buffers of absurd size in order to contain pathnames of unpredictable depth?

Also paths need to stay identifiable. And you would not be able to recognize differences of such overlong identifiers in the middle. Unicode confusables aside

I regularly hit it when using yt-dlp, particularly from twitter where it puts the text of the tweet into the file name by default.

That's more likely NAME_MAX, which is 255.

You could maintain an index for the 2^64 or 2^112 names this allows.

For everage user nothing changes - each folder will just have "thumb.db" with all metadata and long names.

Reply on news.ycombinator.com