https://docs.oracle.com/javase/7/docs/technotes/tools/window...
-encoding encoding
Set the source file encoding name, such as EUC-JP and UTF-8. If
-encoding is not specified, the platform default converter is used.
where that last bit about the platform default converter is something I'd expect from a language that comes out of a squishier (like Python or POSIX) culture.Externally, Java has historically dealt with whatever character set was in use on whatever platform it was running on. As time as gone on, most platforms have converged on UTF-8 for external data storage and interchange. Time for Java to do that too, at least by default.
Which is often not Utf-8 on lots of operating systems depending on how they are configured. Even Linux has issues with this. Even in 2023. Most distributions set a default encoding of UTF-8 of course but if you start your server early in the boot process as root, none of that may be applied. If you write your own systemd scripts, be sure to make this explicit as well. I've found this out the hard way where the default was latin-1 and combined with somebody not doing what they should be doing, you end up dealing with corrupted data.
It's why not specifying a encoding when decoding bytes to a string is a mistake in Java that things like spotbugs check for and warn against. It's one of those things that IMHO should have been deprecated in Java ages ago. Instead they keep on adding more APIs that do the same thing. Removing it would probably break lots of code. But the point is that that code should already be considered broken and should be fixed.
Thankfully, Kotlin uses sensible default parameters in Kotlin extension functions that do similar things for this (of Utf-8). If you want something else, you have to be explicit about it. That's the only sane way to deal with character encodings IMHO. As long as you don't use Java APIs directly, things are fine.
There are all sorts of ways to make this happen: APIs that set a mime type but no character encoding, progams that open and save files, databases that don't default to using utf-8 (or like mysql actually have some non standard 3 byte version of utf-8 that doesn't deal with emojis and a lot of other characters). Linux distributions don't set UTF-8 as the character encoding until late in the boot process. IMHO there's no good reason why it should ever do anything else than UTF-8 in a shell from the microsecond it boots until it shuts down. Changing all that is hard however. Not for technical reasons but for reasons of backwards compatibility with systems that are arguably part of the problem here.
I think the root cause for this might be a blind spot that people in English speaking countries have since they don't use a lot of special characters and therefore don't see the problem. As soon as you start using Spanish, Swedish, German, etc. You are in trouble. And forget about any non latin scripts. Latin-1 is not appropriate for any of those. Utf-8 is a better default that should be usable everywhere. Some countries might prefer utf-16. But that's mainly a memory optimization that should not matter much in 2023.
That is not 100% true anymore. Java 9 (September 2017) changed “the internal representation of the String class from a UTF-16 char array to a byte array plus an encoding-flag field. The new String class will store characters encoded either as ISO-8859-1/Latin-1 (one byte per character), or as UTF-16 (two bytes per character), based upon the contents of the string. The encoding flag will indicate which encoding is used.” (https://openjdk.org/jeps/254)