If the system as designed accepts almost literally any characters in pathnames, it doesn't make a lot of sense to complain about people using "naughty" (troublesome) characters in pathnames they create.
It would have made a heck of a lot more sense to have disallowed stupid characters (all white space and control characters at a minimum) from the outset. However, changing it at this point is pretty impractical.
Of all the criticisms of UNIX, the singular one that strikes me as valid is the way that it accepts unquestioningly insane characters in pathnames. I would also maintain that case sensitivity in pathnames and a lot of other places is also a human interface error. It is arguable, and a matter of taste, but I find it obtuse. You should be able to "say" pathnames. The necessity for circumlocutions like "Big-M makefile" is offensive.
I'm one of the biggest fans there is of UNIX and C and the Bourne shell and derivatives, but recognition of weaknesses and flaws in those you love is not weakness; it is wisdom.
In fairness, though, allowing non-ASCII characters is what enabled the (mostly seamless) transition to UTF-8 filenames.
"So don't do it."
The problem with filenames is just a symptom of the biggest problem of UNIX conventions - passing around unstructured text. Filenames should have one well-defined format (AFAIR kernel allows pretty much anything but the NULL character). That's it. For most applications, filenames should be opaque data blobs compared for binary equality. But because we're passing around unstructured text, each program has to parse, reparse, and concatenate strings via ad-hoc, half-assed shotgun parsers. Each program does it slightly differently, hence the mess.
As long as everyone recognized that putting certain characters in pathnames was counter-productive, things worked fine. Nobody ever dreamed of putting a space character in a filename when they all came from a CLI background. When barbarians came from Mac/Windows to UNIX, they lacked this background, and there went the neighborhood.
I remember being stunned when I first encountered two periods in one filename! But it only took me a few seconds to grasp the fact that period was just another character, "suffixes" were just conventions, and it all made perfect sense. OTOH, the first time somebody showed me a file named "-fr\ \*" and suggested I delete it, I got one of my first disillusionments.
P.S. - "/" is a character which is also "special" in pathnames. Actually, the particular FS may make other exceptions which are not globally enforced by the kernel. ZFS has several (such as "@").
I could buy this if not for the fact that back when UNIX was created, there were already better operating systems and sane solutions to those issues existed. It's more like that those aspects simply weren't really thought through, but instead just hacked together.
Contrary to what seems to be a popular opinion nowadays, UNIX wasn't the first real operating system, just like C wasn't the first high-level programming language. I knew I actually believed the latter, due to the way C/C++ many books were written. But no, in both the worlds of programming and operating systems, there already were better thought-out solutions. It's a quirk of history that UNIX and C ended up winning.
There were differently thought-out solutions, but not necessarily better or worse. Perhaps the issues they thought out didn't matter as much at the time and very unlikely to matter even a little bit today, and who knows how much they got wrong and much worse. It's very hard to speculate. But one thing I'm sure of is that these things cannot be really well designed from scratch and all the problems manifest only once they are used by people. So widely used systems can only be compared to widely used systems, not something niche or unused.
Our industry does seem to be stuck in circles, continuously forgetting the ideas of past cycles and reinventing them, only for them to be forgotten again. To see that phenomenon in action, one does not have to look much further than the last 10-15 years of history of JavaScript to see how the web ecosystem basically slowly reinvented already long established practices from desktop operating systems and GUI toolkits...
Shortly after timeshare systems were created, people were thinking about security.
For those who're interested, there's a pretty good overview of them in Practical Common Lisp[0]. I'll quote the first paragraph of the subchapter "How Pathnames Represent Filenames", which serves as a decent TL;DR:
"A pathname is a structured object that represents a filename using six components: host, device, directory, name, type, and version. Most of these components take on atomic values, usually strings; only the directory component is further structured, containing a list of directory names (as strings) prefaced with the keyword :absolute or :relative. However, not all pathname components are needed on all platforms--this is one of the reasons pathnames strike many new Lispers as gratuitously complex. On the other hand, you don't really need to worry about which components may or may not be used to represent names on a particular file system unless you need to create a new pathname object from scratch, which you'll almost never need to do. Instead, you'll usually get hold of pathname objects either by letting the implementation parse a file system-specific namestring into a pathname object or by creating a new pathname that takes most of its components from an existing pathname."
[0] - http://www.gigamonkeys.com/book/files-and-file-io.html
They also came as a way to handle file paths at all. Generally Lisp prefers other data representations over string representations. The user input (text string) will be parsed into some kind of data.
There is an excellent discussion of the topic[0]. I find it utterly definitive in the way it relentlessly shows how you can't "fix" this issue completely any other way than by ruling out the bad characters by making the kernel disallow them.
[0] https://www.dwheeler.com/essays/fixing-unix-linux-filenames....
IMO the best approach would be to separate between file name and the file object. When I edit a file with vim, should vim really need to know the name of the file? No. Likewise for a lot of other utilities as well. If instead of being so focused on file names and paths everywhere and we operated instead mainly on inodes then I think much would have been won. Now in some instances the file name is of interest to the program itself, for example if you attach a file to an e-mail, upload it with a web browser, tar a directory, etc. but in all of these instances I think that the file name should be more separate and even for most programs that want the file name they should just treat the file name as a collection of bytes that have close to no meaning.
In other words, I would want to translate paths and file names into inodes in just a very select few places and then keep them separate.
This is what I am going to do in my Unix-derived operating system. I will get around to implementing said system probably never but you know, one can dream.
For sufficiently narrow definitions of "bad", sure.
It is probably a bad idea to allow mixed character-set filenames, as that allows homograph attacks[0], and there are other non-control characters like the zero-width space and it's brethren that should be disallowed across the board.
In English you probably also want to disallow ligature characters like fi, ffl.[1]
There are other "good idea" limitations that may affect internationalization of various languages (not in terms of making it difficult, just constraining it, as in the above English ligature example).
For example, it is probably a good idea to disallow Hebrew diacritic symbols[2] like niqqud in filenames.
[0] https://en.wikipedia.org/wiki/IDN_homograph_attack
[1] https://en.wikipedia.org/wiki/Typographic_ligature#Ligatures...
As someone who would be affected by this directly, I can tell you right away this rule would be a no-go. I plainly need the ability to mix Latin and Cyrillic characters in my filenames. A filesystem or OS that wouldn't let me do so wouldn't even be considered.
A very simple rule of thumb is, if it is a title of a book (or a song, or a film etc), it should also be a valid filename.
Same thing with the letter 'т': in cursive, it becomes 'т', which in many (but not all) fonts looks the same as cursive 'm'.
But I find it an unlikely attack vector to begin with. The main concern with homographs is in URLs and other external resources.
Still though, even if we only block some control characters, doing so could lead to problems with future character encodings.
Personally I hope UTF-8 / UTF-16 / UTF-32 is the final set of character encodings but we can't know that it will be.