> The short version of what's going on is that when they put the code into GitHub, GitHub used UTF-8 for encoding, and DOS uses Code Page 437.
So why isn’t this a problem if you’re compiling it under DOS?
So why isn’t this a problem if you’re compiling it under DOS?
I don’t think that’s the actual problem. GitHub does not change the encoding of files in Git repos. Nor (generally speaking) does Git itself [0]
It appears most of the encoding issues were introduced by manually editing files, in response to a corporate policy that certain comments be censored (mainly comments that could be viewed as unprofessional). While it is possibly to do this without altering the encoding of the files, it appears the person who did it used an editor that rewrote the file encoding, and didn’t realise it was doing that
[0] Git can be configured to do automatic conversion between bare LF and CRLF newline formats. That’s generally a bad idea. It is mainly people on Windows who turn it on, although nowadays many Windows tools can handle Unix style line endings without issue, making it largely unnecessary. Also, you can install Git repository hooks that alter file contents as they are being committed, although the majority of repos don’t have any such hooks installed
This problem used to be a whole lot worse under pre-OS X versions of macOS that used CR (just CR, no LF) as the line separator. At least now there are only 2 commonly used conventions and you can essentially just ignore any CRs you encounter in most cases.
That bare CR newline convention used to be very widespread – not just Classic MacOS, also many 8-bit micros (Commodore, Acorn, Apple II, TRS-80, ZX Spectrum, HP-85), Oberon, MIT Lisp Machines and Microware OS-9. But, by the late 1990s, Classic MacOS was the only one of those systems with any mainstream significance. And now you'll only encounter bare CR newlines in retrocomputing or obscure legacy systems – oh, and raw mode terminal input.
That's fair, but I'd argue an equally sensible default would be to respect the conventions of the language ecosystems you're working within, and for Java, Python, Rust, that's LF. IntelliJ provides a lot of configuration on a per-language basis but line separators for new files is a global setting for some reason.
In one of them, I decided to make carriage returns a lexical error. In fact, the only C0 control it allows in source text is LF, it doesn’t even allow tabs (my personal answer to the perennial spaces-vs-tabs debate).
What seems to have happened, is that at some point they opened it as UTF-8 in an editor, which replaced some Cp437 box drawing characters with U+FFFD (encoded as the byte sequence EF BF BD). You can replace these with any single byte and it will build, but making the comments and some TUI stuff look correct requires more editing.
Typical source code doesn't use characters outside that range anyway?