Case-Insensitive Ext4
lwn.net
lwn.net
Having had a fair amount of experience dealing with case sensitive, case insensitive and HFS+ (argh!) file systems in rclone, all I can say is that it is really hard and much better left to user space. Rclone does syncing between diverse systems with different ideas about case sensitivity and normalization when matching up the the source file with the destination file.
I'm not even sure that I agree with normalizing the UTF-8 strings - some users really care that they have two files which look identical but have different encoding, for example files with é and é which evaluate to 0x65+0x301 and 0xE9 respectively but look identical.
I might possibly go for filenames having to be valid UTF-8 - that would at least be consistent with Windows and macOS.
PS I haven't found a way of discovering whether a file system is case insensitive or case sensitive without creating a file - is it possible?
You should be able to do ioctl(EXT4_IOC_GETFLAGS) on the directory and check if EXT4_CASEFOLD_FL is set. I'm not sure if this flag is applied recursively to child dentries (though I would assume it is).
a.txt != A.txt != a.TXT and so on. They're different names of different files.
Next generation *insensitive behavior should then be that cat.txt and neko.txt are the same (therefore only one of them allowed).
So everything is determined by how you interpret directory entries.
> a.txt != A.txt != a.TXT and so on. They're different names of different files.
Only if you define your != operator that way. It’s completely reasonable to have equality under some normalization.
Doesn't Linux/Unix already have such ambiguity? E.g., multiple hard links to the same file; symbolic links; etc.?
If you would make the filesystem case insensitive ls would only show a.txt, but suddenly a "virtual" A.txt a.Txt etc would also exist.
Now there are some folders on my machine that i would like to be case insensitive like ~/Downloads, never having to check if i downloaded distro.iso or Distro.iso would be nice. But i can see so many more problems with files i did not expect to exist and not showing up in my directory listing suddenly being there when i open something "else".
When I was in university this was the argument given by the lecturer of the operating systems course in favour of CI, and I found it convincing. (FTR he is the author of 33 internet standards, and not a fan of Windows in most other ways.)
Losing the ability to have two differently-cased filenames in the same directory doesn't seem like a great loss.
And let's not even start with non-English auditory where you have to render filenames as "С as a dollar, У, another С as a dollar, English T, E, M" (that's "system" btw).
Your mom is pretty good at IT, clearly.
In my experience, this sort of scenario with "normal users" will result in one of the following:
- user opens both files and tries to figure out which one "looks better" for the task at hand. If questioned later, they will likely hide this fact to avoid having to admit their judgement was somehow involved.
- user fires back an email saying "there are two files with that name !! Please do it for me, this is complicated !!11!"
Case sensitivity looks like an advantage only to us nerds. There is a reason many systems go all-uppercase-all-the-time, even where they could easily use casing: it's a big source of confusion. Spoken language has no case, after all.
But if she (or any normal user) only sees 'myfile.txt' or "myFile.txt" or "Myfile.txt", she wouldn't respond with "that file doesn't exist." Instead, she'd be more likely to assume you made a mistake in your specification.
Read through the comments on the LWN article: case insensitivity is a practical problem because user-space implementations (such as samba) must perform an O(N) check across all files in a directory before confirming that a file doesn't exist. At the same time, existing programs that assume case-insensitivity (coming from the Mac and Windows worlds, mostly) are often none too consistent about creating 'foo.txt' but opening it (potentially from another program) as 'FOO.TXT'.
Why? Human languages are also case-sensitive (e.g. 'moon' and 'Moon' have different meanings).
Nowadays, casual users rarely enter filenames by name, and for output, case-insensitive filesystems are still case-preserving (and users expect that).
> Losing the ability to have two differently-cased filenames in the same directory doesn't seem like a great loss.
It is not really about this ability, it is about plenty of direct implementation and usability issues that it causes, like:
- it makes everything encoding-dependend (which complicates e.g. handling of mixed encoding filenames and filenames with invalid utf-8 sequences)
- interoperability with case-sensitive systems (unpacking archive containing files differing in case)
- backward compatibility (new unicode version adds new characters and their case mapping)
Considering that case-sensitivity is not the only such issue (there is also different unicode normalizations and homographs) it seems to me that it make sense to have FS code and low-level utils encoding-independent (and therefore case-insensitive) and do case sensitivity (and perhaps. homograph handling) only in high level user code.
You'd know I meant the same thing if I wrote "go to the moon" or "go to the Moon". If you wrote "Hi ian" and I replied "There's nobody here by that name, can't imagine who you might mean" you'd think I was being obtuse.
set completion-ignore-case On
Personally, I’d prefer an obtuse computer over one that makes assumptions and is occasionally wrong.
Absolutely, but it's where I find people get tripped up who aren't as used to tech. The thing I have told the computer to do is technically wrong in a minor way but it's extremely obvious to anyone that looks at it what I want. There are multiple reasons why we usually want this to be the case (simpler to implement, cost of returning an error is low but the cost of automatically doing the wrong thing can be huge, etc).
The rules aren't universal either. Some machines are fine with different casing, some aren't. Email addresses aren't case sensitive, typically, but my user name might be. Email addresses have to be exact, except with gmail where . isn't important, and anything after + is ignored. Google search from the omnibox will assume what it thinks you wanted to search for by default but urls entered in the same place are extremely picky.
If I have "bài tập #14.pdf" and "bai tap #14.pdf" in the same directory, basically everyone is going to expect them to be the same thing, not different things. Trying to explain they are different because filesystem developers only understand English-centric conventions isn't exactly easy to explain either.
There are bigger problems, though - currently Linux allows having multiple files with the exact same file name, differing only by UTF-8 normalisation.
Having a unique name is uncommon.
In the current paradigm where files are predominantly accessed by a path and filename, yes. But that need not be the only nor the best way of referring to persistently stored data.
One incontrovertible justification for case-folding is to support Samba shares efficiently. Samba has to implement case-folding to clients and if it is not done in the filesystem datastructures you have to read in many directory entries to figure out if one would match with case-folding.
There is plenty of MS Windows and MacOS software that just expects case-folding from the operating system.
The article is old and this was released in 5.2. It can be turned on per (empty) directory with chattr https://kernelnewbies.org/Linux_5.2#Optional_support_for_cas...
Talk about a confusing experience. As soon as you checkout the file, it tells you that a file has been deleted. All normal git ways of fixing the issue don't work, because it comes down to the git repo thinking there are two files but the file system not letting that happen.
It caused an issue that only one of us in the team experienced, since they had a case-insensitive file system, I think Windows. The reverse also happened, where everyone but that person saw an issue, due to their case-sensitive file system.
In the end, "git mv" solved those cases, but I never fully understood how to avoid them occurring in the first place. I think we all just agreed on a convention of all lowercase folder/file names.
But my mother, case-sensitive filenames on her MacBook would make her life much worse. She can barely manage the high volume of files she creates as a teacher as it is, and the potential for accidentally doubling that number, or worse, is terrifying.
[0] So of course I have an alias `mvd filename.that.should.be.in.something.like.proper.case.mkv Filename.That.Should.Be.In.Something.Like.Proper.Case.mkv` that actually does `mv filename.that.should.be.in.something.like.proper.case.mkv Filename.That.Should.Be.In.Something.Like.Proper.Case.mkvzzz && mv Filename.That.Should.Be.In.Something.Like.Proper.Case.mkvzzz Filename.That.Should.Be.In.Something.Like.Proper.Case.mkv` and all is well.
Whether the lowercase version of "I" is "i", that depends on your language. Often it is, but for example in Turkish, lowercase "I" is "ı", and lowercase "İ" is "i".
How will the language be specified on the case-sensitive filesystem? Per file, per directory, per device?
Can I have English "I.txt" and Turkish "i.txt" in the same directory? Are they compared using the English rules or the Turkish rules?
What happens if I copy a file to a directory or device using a different language? To a device using a different filesystem that doesn't support languages? If I copy 10 files from a Turkish computer to a memory stick, then copy them to an English computer, will I get only 9 files at the end, because "I.txt" and "i.txt" got overwritten?
Also note that once you set up the case sensitivity status on a directory/device, it could be hard to change it. For instance, if I have two files named "I.txt" and "i.txt" in a directory that is case-sensitive, I can't change that directory to case-insensitive using English rules. But I could using Turkish rules. You would have to do a full scan of the area you are planning to change in order to check whether anything would clash.
Also, you have the problem of filename searches. At the moment, it is possible to open a file by name in a directory that has lots of files, and the filesystem can do an indexed search for that file. If you change it to case-sensitive, then either you need to do a linear search instead (which could absolutely cripple some use-cases), or store an indexed case-insensitive lookup table, with the additional proviso that the index must be able to give you back the canonical filename. If the case-sensitivity changes, then this index must be recreated from scratch.
Another example of this is the German eszett ("ß".upper() == "SS" but "SS".lower() == "ss" != "ß").
(Reading this discussion thread feels like some strange alternative reality where "take ... from my cold dead hands" is transported to FS world.)
https://docs.microsoft.com/en-us/windows/win32/api/fileapi/n...
Because everything on windows used the win32 api applications are friendly to case insensitivity but you can still run into problems interacting with other systems that are not.
https://www.hanselman.com/blog/GitIsCasesensitiveAndYourFile...