Hacker News Reader

Comment by flohofwoe | Hacker News Reader

> non UTF-8 encoded strings on input/output

UTF-8 on stdin/stdout works perfectly fine (unless you are on Windows of course, which is stuck in in the early 90s when it comes to international text encoding).

> Using LF or CR or CRLF as line terminators

This is also an operating system convention, and it would be better if programming languages wouldn't try to "guess" the correct line endings, since this causes more problems than it solves - but again, this is mostly a Windows specific problem, and it's Microsoft's job to finally bring Windows into the current century.

View on HN

No, it was an Apple, Unix, and Microsoft problem.

Unix used LF, Apple used CR, Microsoft used CRLF.

They are all ASCII carriage movement codes, which is about driving the paper feed and print head of an ASR-33 or equivalent.

So they all made the "wrong" decision about what to store in a file.

They just chose different wrong characters.

> They just chose different wrong characters.

Unix followed Multics. Multics chose right. ASCII/EMCA-6/ISO646 drafts discussed this at least as early as 1963¹: “For equipment which uses a single combination (called New Line) [...] NL will be coded at FE₂ [Field Effector 2 = 0x0A].”

¹ doi/10.1093/comjnl/7.3.197

For an OS that was being created specifically to process text, having the equivalent of CR being separate to LF to allow for overprinting would/should have been a requirement.

I'd say Multics/Unix was technically correct, except this was still the wrong decision for I/O ever since.

The Record Separator is the logical character code to use to indicate the end of a line of text and print position characters, assuming that a line of text is a "record".

> Apple used CR

Apple hasn't been using CR since the release of OSX (26 years ago). Microsoft could have made the switch at any time too (just as they could have switched to UTF-8 as universal text encoding on Windows), they just choose not to.

In the end it's not the job of programming languages to clean up Microsoft's mess ;)

We're literally talking about two decades before that.

The switch sure sucked though. I doubt Microsoft would risk their reputation for backwards compatibility.

> In the end it's not the job of programming languages to clean up Microsoft's mess ;)

Why is it Microsoft's fault? They just stayed on their legacy implementation, Linux and Apple chose to move from the legacy implementation to another legacy implementation. That seems dumb.

Both Linux and MacOS followed the Unix implementation, both of them are derivatives of Unix, so why would that change? Unix derived from Multics which chose LF.

The issue is that none of the print carriage movement ASCII characters should be used internally to indicate "end of line", because each of the chosen possibilities are used separately to indicate different carriage movements.

The logical decision would have been to choose one of the "separator" characters to indicate "separation of one line from another" and then allow the I/O drivers to decide what to send/receive to/from a particular device.

UNIX's LF precedes them by at least half a decade, probably more.

CRLR is Baudot, predating UNIX by what, a century ?

Apparently it's from 1901 (Murray code) or 1932 (ITA2).

The fact that both Apple's and CP/M codes came out roughly at the same time, both on microcomputers, shows that it was probably just a design decision.

rswail said ASCII, which definitely pre-dated Unix, not the other way around. And there was some to and fro about the equivalence of LineFeed and NewLine in the 1960s.

I think PCDOS/MSDOS copied CP/M's use of CRLF for line separator.

Some believe Gary Kildall picked CRLF for CP/M since he used DEC TOPS-10 to develop CP/M. see https://www.quora.com/Why-did-CP-M-stick-with-the-CR-LF-stan...

What programming languages try to guess line endings? Or are even aware of them?

Ok, technically not the programming languages, but their stdlibs. On MSVC at least, opening a file in text mode via fopen will translate CRLF into LF on read, and LF into CRLF on write, which has been a neverending source of confusion since at least the 1990s.

Reply on news.ycombinator.com