Comment by cosarara | Hacker News Reader

Comment by cosarara | Hacker News Reader

Parent Full thread

It sounds like this would be a nightmare in Unix filenames; even if everything in my system is perfectly clean utf-8, I could be trying to open "café.txt" and get a No such file or directory error because the input has é encoded a different way from what is saved on disk. You would need to list the directory contents, decode everything, perform the comparison in Unicode code points, and pray there is no more than one match. Any web UI that deals with files would have this kind of issues.

Yeah. Though it really depends on the system.

Some older filesystems, like HFS+ would normalize filenames. In APFS Apple decided not to normalize file names, but this caused a lot of issues. Later they have made APFS normalization-insensitive (so you cannot store two files with the same name, but different normalizations). Different normalizations are handled transparently by macOS frameworks.

On Linux with btrfs:

    $ touch schön
    $ cat $(echo -e "scho\u0308n")
    cat: schön: No such file or directory
    $ touch $(echo -e "scho\u0308n")
    $ ls -l
    total 0
    -rw-r--r-- 1 daniel daniel 0 Dec 17 12:00 schön
    -rw-r--r-- 1 daniel daniel 0 Dec 17 12:00 schön
    $ ls | hexdump -c
    0000000   s   c   h   o 314 210   n  \n   s   c   h 303 266   n  \n

Unicode, loads of fun :).

Edit: updated to make this copy-pastable.

I wonder if ZFS falls prey to the same issue. I've heard good things about it otherwise.

Reply on news.ycombinator.com