Comment by rswail | Hacker News Reader

Comment by rswail | Hacker News Reader

It's a damn shame that the control characters for "Field Separator" (FS ASCII 28 decimal) and "Record Separator" (RS ASCII 30 decimal) weren't used instead of TAB and CR/LF.

Then the whole CRLF vs CR vs LF for line endings in files would have been totally avoided, with CR returning the print-head/cursor to the left-most position (right-most on R-L languages) and LF literally moving the print-head/cursor down.

The drivers for the particular terminal could have translated as required, and we would not need (in 2025!) to be specifying to git and elsewhere what to convert line endings to/from.

I know that back in the days of literal TTYs (ASR-33) where I cut my teeth, the BEL/BS/SP/CR/LF/FF etc characters were literally controlling a print head, but even then it didn't make much sense that we didn't have internal representation of what was needed (field separator, group separator, record separator) that was translated when doing actual I/O to a physical device.

> It's a damn shame that the control characters for "Field Separator" (FS ASCII 28 decimal) and "Record Separator" (RS ASCII 30 decimal) weren't used instead of TAB and CR/LF.

No, it's not.

That stuff is left over from 1940s-1950s schemes for polled multidrop teletype machines - multiple stations on one wire, taking turns. The idea was that you punched paper tapes locally and then put them into a reader. When it was your station's turn, the central machine would send a polling sequence of a few characters, the selected reader would start up, read one message into the upstream system, then stop. In the other direction, the central station could send a turn-on code to any machine, print a message, then turn it off. Only one machine at a time could run. Those control characters were the delimiters for that process. All of this was mechanical, done by a unit called the "stunt box", which you can look up if really bored.

RS-485 is a descendant of that technology, still used in some industrial control systems.

(I used to restore antique Teletypes for fun, and had a functional Telegraph Office at steampunk conventions for about seven years.)

Why? The use of TTYs for computing interfaces is very different to using them as teletypes on comms links.

I thought the control characters for that sort of use were the DC1-4 characters?

RS-485 uses polling but my understanding is that the characters for polling usually involve like "FF" (hex) and some sort of device ID.

I just read a TTY model 28 manual about the stunt box and I didn't see anything specific about FS/GS/RS there.

How were messages delimited in the polled multidrop environment?

TTY interfaces initially were teletypes on comms links.

Yes, but they were character based, not message based as being discussed.

The ones I used were ASR-33s on 20mA current loop interfaces.

Modern protocols running over RS-485 UART usually use some kind of HDLC-inspired framing scheme with flag characters and byte stuffing.

But still there is a lot of stuff that uses ASCII STX/ETX and then some kind of field separators inside otherwise human readable message. Things like industrial scales, industrial barcode readers and what not usually use something like that as default output format.

> I just read a TTY model 28 manual about the stunt box and I didn't see anything specific about FS/GS/RS there.

The Model 28 was a 5-bit Baudot (ITA2 code) machine with a stunt box. The Model 35, an ASCII machine, had a similar system, with different coding.[1] The term "Selective calling" replaced "stunt box".

ASCII-67 is modern ASCII, but there was previously ASCII-63.[2] That keyboard has special keys for selective calling. WRU, RU, EOT, EOA, TAPE, ~TAPE, etc. Codes didn't become standardized industry-wide until ASCII-67. Before that, there were lots of variations.

Here's a popular view of the process in the Baudot era.[3] That big switching center is essentially Sendmail built from of relays and paper tape equipment. It was called "Plan 55-A".[4]

[1] https://www.smecc.org/teleprinters/35selcall001.pdf

[2] https://lostbits.net/blog/the-teletype-model-35-character-se...

[3] https://youtu.be/PKJyGuhCQdQ?t=369

[4] https://en.wikipedia.org/wiki/Plan_55-A

Here's a pdf of a blank prop Western Union telegram form. Have you got others? https://www.hplhs.org/resources.php

I've used their fonts. Our telegram blanks were very similar, but custom.[1]

[1] https://aether.ltd

Editing a file like that with a text editor is a right pain. Inserting these characters isn't all that straight-forward, and everything will be on a single line. In theory RS could be treated as a newline, which will only add to the line ending confusion.

I think this is one of those things that sounds kind of nice in principle, but where the real-world practicalities just don't work out.

Text editors should be programmed to support those separators. Would've saved us from CSV.

The file editing is an I/O issue, not a character issue.

The I/O drivers could translate as necessary, that's the point of using an explicit character to mean "new line" as opposed to either "carriage return" or "line feed".

Effectively moving away from explicit carriage and paper controls to a character that was explicitly internal to mean to generate whatever carriage/paper controls were needed on output/editing etc.

Wouldn't we still need a "Line Separator" character so we can include line breaks in fields and records for formatting purposes?

RS separates logical "lines" or there is "Group Separator", but for internal line separators in a multi-line text field, the CR and LF characters could be used, because they have "standard" meanings for text:

CR: move next character position to position 1 horizontally LF: move next character position to vertically below current character position, potentially inserting SP characters.

So a combination of CR/LF, or LF/CR would have the effect of "New Line".

Also CSV files would not exist.

Comma Separated files would likely exist because they're human readable, whereas FS/GS/RS/CR/LF are not visible.

There is "pipe delimited" which used "|" (ASCII 124 decimal) to separate fields, which actually makes way more sense, because that character is rarely used in normal text.

It's also a good separator when a human is reading the file.

They could be made visible. On MSDOS it was common to have editors show an arrow as ESC or EOF, respectively pointing left or right. A lone CR was a musical note.

I am not sure whether RS could have replaced LF or CR+LF in this kind of free-format database; overloading record and line separators makes sense. But if FS-separated fields were a thing, people on MSDOS would have been used to seeing ⌙ as a field separator and likewise for other OSes.

I prefer alt 1 (smiley face) as a field separator. It definitely is rarely used in text.

But we do NOT need to setup CRLF translation at all? Who in his sane mind uses CRLF even under windows? All my source codes are in UNIX format (aka LF only) and I use it consistent under Windows, Cygwin and UNIX.. Why make your life harder?

I think the issue is that people don't make their own lives harder, but sometimes they make the lives of other people harder (often unintentionally). In my experience, the CRLF files I run into tend to come from developers who are only working in Windows and either aren't aware of the issue or don't particularly care to do anything about it. It isn't something I run into often when cloning a git repo, but having recently started dabbling in the world of mod development for games, there's a lot of stuff out there that seemingly has never been touched by someone who's used Linux before. To be clear, I don't blame anyone for this, since people working on things for free in their spare time shouldn't have to worry about anything other than what interests them, but it's a bit amusing to me that I sometimes have less to configure to run an .exe on Linux than I do to start editing the source for it in a way that makes it easy for me to merge any new changes that happen upstream.

> I sometimes have less to configure to run an .exe on Linux than I do to start editing the source for it in a way that makes it easy for me to merge any new changes that happen upstream.

Are you doing anything more than calling dos2unix?

The language runtime will insert/remove CR for you if you open the file in text mode on Windows. You'd be surprised how much stuff cygwin itself actually does for you. If your code isn't too Linux specific try building it under mingw instead and you can see what's actually going on.

Cygwin does NOTHING under the hood, because I asked it to do nothing. I am aware of text mode of open() under Windows/Mingw. Im also aware of possibility to do CRLF <-> LF translation under cygwin. I never liked it, and I went to LF only quickly. Hence, I use it everywhere and live is easier :)

> Who in his sane mind uses CRLF even under windows? All my source codes are in UNIX format (aka LF only) and I use it consistent under Windows, Cygwin and UNIX.. Why make your life harder?

Why would you use LF if the focus of the application that you develop is Windows? You make the life harder for these users. :-)

Seriously: your argument typically comes from developers who consider GNU/Linux as a first-class citizen as a development or deployment platform, and Windows only as a second-class citizen (assuming that a Windows port actually exists).

I can accept an argument like "I, as a developer, don't care about Windows and its users, so all my source code is in UNIX format." This is a conscious, though quite political decision.

But an argument like "not using LF makes lifes harder" without a relativization on which premises this claim is build (such as "GNU/Linux and macOS users are much more important for as, and if there exist any users or developers who use Windows, we consider them to be undesired, because they make everybody's lifes harder") is intellectual ignorance.

Using LF on Windows is really not difficult. Any text editor actually worth using supports both types of line endings - even Notepad has handled this correctly for years now, from what I remember. I've never been caused any inconvenience as a Windows user by files with LF endings. I never use CRLF if I can avoid it; I only will if working on a codebase that already uses it throughout.

Oh, its simple. There are many many flavors of UNIX and only one Windows. Hence, I see LF as more portable way to store source code, thats why it is prefered. Additionally, im heavy CLI user so again, more geared toward UNIX systems.

If you develop on windows and only for windows, use CRLF all the way. Just do NOT forget to set .gitattributed correctly, and you are set.

> There are many many flavors of UNIX and only one Windows.

With ReactOS, there even exists a flavor that is not developed by Microsoft. If you consider Wine and its derivates as an implementation of the WinAPI, you have one additional (or multiple if you consider their derivates) implementation of the WinAPI.

Even if you only consider the flavours of Microsoft Windows that are full operating systems, you immediately get multiple ones:

- the discontinued Win 9x series

- the discontinued Windows CE, Windows Embedded CE, Windows Embedded Compact, Windows Mobile, Windows Pocket series (technically quite different from both Windows 9x and Windows NT)

- Lots of variants (indirectly) derived from Windows NT:

* Windows (desktop OS)

* Windows Server

* Windows IoT (I would claim that at least for the Windows 10 IoT Core version, the user experience is quite different from both desktop and server Windows)

* Windows PE [1]

* Further discontinued variants such as Windows Phone

[1] https://en.wikipedia.org/wiki/Windows_Preinstallation_Enviro...

The only valid reason you mention here are legacy Windows, like 9x series. Anything NT based is happy with LF only if you have right tools.

I never touched ReactOS, but I suspect is modern enough to handle LF only too. It was targeting Win2003 AFAIR.

So yeah, I stand corrected abit. If you develop only for windows (especially legacy), use CRLF. Anything w/ portability in mind, go for LF only...

The ability to do that is a recent development. And numerous programming environments still default to adding CRLF and stripping it off when running on Windows, and all the system tools on Windows generate files with CRLF as line terminators so all software needs to deal with them.

> Who in his sane mind uses CRLF even under windows?

Hopefully nobody is using it for documents, but most text-based internet protocols, including SMTP and HTTP, use CRLF for line endings.

I fully agree, but we're talking history here, before Unix existed.

The confusion about "LF" (Unix) vs "CR" (Apple) vs "CR/LF" (MS/DOS & Windows) is because we didn't have an explicit "Line End" character in 7-bit ASCII.

There was "EOT" (Ctrl-D/ASCII 4/EOF in Unix) , "ETX" (Ctrl-C/ASCII 3), and other characters that were used to mean other I/O, including ones that didn't match the ASCII definition like Ctrl-S/Ctrl-Q to stop/start I/O, and Ctrl-Z to mean EOF (MS/DOS & Windows).

Reply on news.ycombinator.com