You can say that about every single design decision made about every product.
The gripe about this particular feature seems misplaced because almost all users will want the sort that's offered and the actual alphabetical sort is likely the desire of a more advanced user who, in fact, is offered a choice through registry editing and/or using a more advanced cli option for the occasion they might need an alternative sort.
This is a sensible default.
No, that's not true. Many aspects of my computer's UI are user-configurable.
How do capital letters sort relative to lowercase letters? How do letters sort relative to digits? How do you consider code points that can correspond to different letters in different lettering systems with different ordering? How do you handle diacritics? Do you want the behaviour to be stable through Unicode normalization? Should it differ based on the character encoding? Should different representations of the same character, such as blackboard lettering or circled numbers, be sorted with other representations of the same character or grouped separately?
You can come up answers for these questions, but there’s no unambiguously correct option. The least subjective option is sorting based on encoded byte representation (if that is even specified), but that is not “alphabetical” and would not be intuitive to most users.
And I would say the reasonable way to define character is grapheme cluster and yes you want it stable to normalization and encoding.
How capital letters/diacritics/different representations affect the order of your alphabet, and which ones are considered equivalent, is something without a clear answer. Same for whether letters or numbers come first, and where punctuation goes. But you don't need consensus on that to fix the problem in the post.
I guess your position is coherent, but it’s very silly.
(I did not downvote you.)