However, Powershell (or more often the host console) has a lot of issues with handling Unicode. This has been improving in recent years but it's still a work in progress.
UTF-32 / UCS-4 is fine but feels very bloated especially if a lot of your text data is more or less ASCII, which if it's not literally human text it usually will be, and feels a bit bloated even on a good day (it's always wasting 11-bits per character!)
UTF-8 is a little more complicated to handle than UTF-16 and certainly than UTF-32 but it's nice and compact, it's pretty ASCII compatible (lots of tools that work with ASCII also work fine with UTF-8 unless you insist on adding a spurious UTF-8 "byte order mark" to the front of text) and so it was a huge success once it was designed.
For example, this Chinese Wikipedia page is almost twice as big in UTF-16 as in UTF-8, because the vast majority of the content seems to be HTML tags:
$ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | wc -c
1877366
$ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | iconv -f utf-8 -t utf-16 | wc -c
3345724
$ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | tr -d '[:print:]' | wc -c
311226
$ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | tr -d '[:print:]' | iconv -f utf-8 -t utf-16 | wc -c
213444
$ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | tr -d '[:print:]' | gzip -9c | wc -c
98373
$ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | tr -d '[:print:]' | iconv -f utf-8 -t utf-16 | gzip -9c | wc -c
89031
I can't find the more detailed article, but this summarizes it: https://techcommunity.microsoft.com/t5/sql-server-blog/intro...
From what I can remember, UTF-8 consumes more CPU as it's more complex to process, has space savings for mostly ascii & European codepages, but can significantly bloat storage sizes for character sets that consistently require 3 or 4 bytes per character.
Big endian or little endian?
My team managed a system that did a read from user data, doing input validation. One day we got a smart quote character that happened to be > U+10000. But because the data validation happened in chunks, we only got half of it. Which was an invalid character, so input validation failed.
In UTF-8, partial characters happen so often, they're likely to get tested. In UTF-16, they are more rarely seen, so things work until someone pastes in emoji and then it falls apart.
UTF16 is really not noticeably simpler. Decoding UTF8 is really rather straightforward in any language which has even minimal bit-twiddling abilities.
And that’s assuming you need to write your own encoder or decoder, which seems unlikely.