Using non-ascii characters in file paths, toolchain config files, and other non-display contexts is just asking for trouble, even if it is your name...
But it's a shame.
In Europe, we do have a lot of non-ascii characters everywhere. Ubuntu puts a "Vidéo" and a "Téléchargements" directory in my $HOME because I'm french. If I were to use my name as my username I would have even more troubles.
I'm careful with not using special chars in names for work, but it feels like I'm a girl trying to not dress sexy in the wrong part of town: necessary, but I shouldn't have to do this, and it's definitely the others to blame.
All in all, I thank the Gods of encoding for Python 3 unicode handling. Having a scripting language that does the right thing out of the box is wonderful on this side of the pond.
Want to offer Unicode validation? Sure having that as an OPTION is fine. Forcing it means I can't rely on that tool to handle real world data which happens to not be valid but is still a valid file-system address.
- if you want to treat paths like unicode strings, you can. Which is great for simple scripts where you don't want to deal with complexity. And 99% of the time, it's enough with modern OSes.
- if you want to threat path as bags of raw bytes, you can. Which is necessary to transparently copy and do not evaluate, as you said, for covering edge cases.
- if you need to actually deal with those as strings but don't want to loose data for edge cases, so a mix of the 2 above, you can use surrogate escapes
Python 2's unicode model was _closer_ to correct, the trivial coercion between byte[] and Unicode.
Conversion also shouldn't imply, force, or check Validation nor Normalization. Labeling a bytestream with an Encoding and validating / normalizing that encoding should be options. Operations on bytestreams with encoding related attributes should set them to either 'unknown' result or to a proper output type if they're aware the manipulations will still yield a valid encoding.
Normalization is more complex, since Unicode strings can be normalized in different ways, then combined, and still be a valid string but no longer uniformly normalized.
There's no excuse for actively supported, paid products to have these problems today.
Algol-68 supported localized sets of keywords; fortunately this language is gone.
You can #define non-ASCII stuff in modern C++. It's your best chance to "localize" a mainstream language.
Same would work for Clojure, but Lisp uses a lot of quirky abbreviations like `cdr` or `setq` that give awkward translations.
https://en.wikipedia.org/wiki/Non-English-based_programming_...
Not saying that this is OK, just explaining why using non-ascii characters, in this day and age, is still asking for trouble.
Windows 2000 is when the OS changed to UTF-16 by default. Before that Windows NT was UCS-2, IIRC only the DOS-based Windows versions were Windows-1252 internally, starting from Windows 1.0. So while ł wasn't supported in Windows 1, characters like ñ were. Windows has literally NEVER been an ASCII-based OS.
Basically - I agree: This shouldn't be a problem, and 7 months is a long time to wait for a basic fix. But there are a lot of footguns hanging around in windows code with respect to character encodings.
Just looking at the first result on google for "c++ get windows home directory" shows this: https://docs.microsoft.com/en-us/windows/win32/api/userenv/n...
Which takes a long pointer to tchar string (LPTSTR) - so this behavior is dependent on the unicode settings of the project at compile time, even today.
The documentation is simply wrong, GetUserProfileDirectoryA which you linked always takes a LPSTR (always "ANSI") while GetUserProfileDirectoryW always takes a LPWSTR (always WTF-16). This is reflected in the function prototype at the top. Only the define GetUserProfileDirectory switches between these two. The define is a compatibility hack and arguably was a mistake, but you can always the W-suffixed function no matter what the project settings are.
Paths are UTF-16 + unpaired surrogates, so a Windows path isn't legally representable in UTF-8.
"Asking for trouble" is a key part of testing. My suggestion would be for a QA person to have their username (and root folder of the testable project) to start with a space, and be followed by an accented letter, tab-symbol, apostrophe, an emoji, followed by an unicode RTL control character and some Arabic text.
People have taken this to a ridiculous level. Giving practical advice about how to avoid problems is not blaming the victim. Telling my child to look both ways before crossing the street is not blaming him if he gets hit. It's not wanting to see him get hurt when the person actually at fault fucks up. Telling someone to avoid non-ASCII characters because a program can't handle it is not blaming them....
Your comment was unhelpful. Great! Not their fault! What now? It's also part of a larger trend that will lead to people being hurt.
My name appears differently in my passport, on plane tickets (not always using the same modification), and my green card. And for the latter two I left out the part of my name that can’t properly be represented at all in ASCII. And you are saying that somehow I am at fault?
Yes... The computer is at fault for not supporting proper names... That's literally what everyone has said. Nobody blamed the user....
> Your attitude reminds me of the people in the 60s who included punch cards in the utility bills, marking them “DO NOT FOLD, SPLINDLE, OR MUTILATE” (it became a meme).
The problem should be fixed, but it's not victim blaming to tell someone how to still submit their bill.
> And you are saying that somehow I am at fault?
No... I'm literally saying the opposite.... I think you need to reread your comment, then my comment.
Telling you to omit those characters so you can still travel internationally is not blaming you in any way shape or form... Your criticism of practical advice being victim blaming is harmful and unhelpful.
As much as I wish we lived in a better world where name characters were better handled, using anything outside of [a-ZA-Z]{12} as a username is a world of hurt. Some people just realize it later than others.
So yes, you shouldn't think of your handlename as your name, it's just another identifier, and choosing simple handle names is a life skill at this point.