Stop Requiring CRLF Line Endings | Hacker News Reader

Stop Requiring CRLF Line Endings

fossil-scm.org

35 pointssmartmic37 comments

> HTTP → RFC-2616 says in section 19.3 says "we recommend that applications ... recognize a single LF as a line terminator...." In other words it is perfectly OK for an HTTP client or server to accept CR-less HTTP requests or replies. It is not a violation of the HTTP standard to do so. Therefore they should.

The most up-to-date version of HTTP/1.1 spec is RFC 9112, which says:

> Although the line terminator for the start-line and fields is the sequence CRLF, a recipient MAY recognize a single LF as a line terminator and ignore any preceding CR.

"MAY", of course, is different from "MUST" or "SHOULD", so I feel like the author's claim that implementations rejecting bare NLs are broken is at odds with the specification.

This comes down to Postel's Law; they recommend liberally receiving what you conservatively cannot send. Also from RFC 2616 but not cited by the author:

> This flexibility regarding line breaks applies only to text media in the entity-body; a bare CR or LF MUST NOT be substituted for CRLF within any of the HTTP control structures (such as header fields and multipart boundaries).

They aren't going to allow sending LF until at least one bump to a higher protocol version where every server MUST accept it.

LF won over CRLF, little endian over big, UTF-8 over UTF-16, spaces over tabs - the world converges but at the same time new frontiers arise and new battles will be fought.

Spaces over tabs is one I don't understand - it's the most democratic way to format code (and it saves redundant bytes of code).

Momentum wins over superior solutions all the time. Just like SMTP, I feel like that tabs vs. spaces is a lost war.

nope

The widespread adoption of code formatting tools has rendered the very argument somewhat pointless. I don't even know which identation most of my code uses - my editor autoformat takes care of that.

In a world where tabs win, you then have a battle over how far to move the cursor which each tab.

no, because then everyone can set it to display how he likes

In the tabs world this is just a style set by whatever displays the code, eg. the user's editor setting.

In the spaces world, you have the battle over how large indentation should be and everyone has to live with what someone else chose.

it amazes me to still see people making the same incorrect and long-debunked arguments, 20 years later. It's almost like these spaces people don't bother actually looking at the options.

For me, there is one single reason spaces are superior to tabs, and that reason is the most important: it ensures that you, the reader, are viewing my code exactly as I, the author, wrote it.

Consider this snippet, where tabs (⇥) equal four spaces (·) in my editor:

    if (someCondition) {
    ⇥   Foo foo = SomeClass::SomeFunctionCall(myParameter1,
    ⇥   ⇥   ⇥   ⇥   ⇥   ⇥   ⇥   ⇥   ⇥   ··myParameter2,
    ⇥   ⇥   ⇥   ⇥   ⇥   ⇥   ⇥   ⇥   ⇥   ··myParameterN);
    }

Looks good.

Now you open this snippet in your IDE, where tabs are set to two spaces:

    if (someCondition) {
    ⇥ Foo foo = SomeClass::SomeFunctionCall(myParameter1,
    ⇥ ⇥ ⇥ ⇥ ⇥ ⇥ ⇥ ⇥ ⇥ ··myParameter2,
    ⇥ ⇥ ⇥ ⇥ ⇥ ⇥ ⇥ ⇥ ⇥ ··myParameterN);
    }

This matters more when the presentation is important, like I might write ASCII diagrams in my code, are have a large matrix with elements aligned, or similar.

Once you're not purely using tabs to indent but also throwing spaces in there, why wouldn't you do it like this?

    if (someCondition) {
    ⇥   Foo foo = SomeClass::SomeFunctionCall(myParameter1,
    ⇥   ······································myParameter2,
    ⇥   ······································myParameterN);
    }

This is the patrician way. Tabs for indent, spaces for alignment. Looks the same for everyone, no matter how big or small they want their indents to look!

Fixed pitch fonts and ASCII art are outdated; not even Dwarf Fortress relies on them anymore. You can sort of line up numbers, but you have to be careful to use U+2007 FIGURE SPACE (not U+0020 SPACE) and U+2212 MINUS (not U+002D HYPHEN-MINUS) to match the size of an Arabic digit. Code indentation only needs to be visibly bigger or smaller, and lining up words and formulae are jobs for TeX.

If you want the parameters to be indented the same amount (why you'd want that, I don't know, but whatever floats your boat):

  blah(
    blip,
    blop,
    bloop
  );

And anyway, 'you see the code the exact same way I saw it when writing it' has the following problems:

1. It doesn't help because you were writing the code, I have to read it.

2. It doesn't help because the way my brain works most likely isn't 100% interchangeable with how yours does.

3. It may even not be true because I may be using a different editor, a different editor theme, a different font, or, by golly, an entirely different monitor resolution.

That being said, I will agree with my sibling comments which recommend the combination of tabs for indentation and spaces for alignment.

People who indent like that should be in jail.

I'm following a junior who is working on a project where everything, including braces, is indented like that.

Some lines overflow on a 2k screen.

So stupid... And it requires much more effort than just indenting logically depending on the depth of the current block. You have to wonder why people do this to themselves and to their colleagues.

The solution to this is to NEVER align, and use tabs for code indentation. And if you do ASCII diagrams, do them in comments and align them with spaces (of course). Tabs ONLY from the start of a line until code. No mixing of tabs and spaces.

Please let go of that childish esthetic to align certain stuff with spaces.

> the reader, are viewing my code exactly as I, the author, wrote it.

That's not a good reason, there are way too many writers with poor design sensibilities, so flexibility is better.

And in your example you'd use elastic tabstops so that a single extra tab would visually expand to align arguments for you if you like it, but not for me if I don't

Indeed, it's a terrible reason.

Why should you have control over what indentation width I choose to use?

There are a bunch of reasons why I might want a narrower or wider tab width. Most obviously, screen real estate - I might be coding on a tiny device.

When you do indentation and alignment properly (tab for indentation, space for alignment, as others have said), it's formatted the way the writer wants, while also respecting the reader's preference for tab width.

Spaces for indentation is incorrect, pure and simple. And the war is not "won".

Alternatively don't introduce incompatibilities in decades old formats. Even if you can somehow convince everyone to accept \n, older software exists and this will only serve to further erode how well things work together. If there was a good way to go back to these format and fix things, the line ending would probably be pretty low on my list.

> Nobody ever wants to be in the middle of a line, then move down to the next line and continue writing in the next column from where you left off. No real-world program ever wants to do that.

I routinely do this while drafting text and tables, and isn't this akin to what multi-cursor input achieves in many text editors?

You are missing a subtle issue: let's say you want to type the numbers 1-3 above each other. If you send 1NL2NL3 the result will be a set of stairs

  1
   2
    3

Since after typing the digit the writing head is behind that character. You have to add a backspace to go back one column. 1NLBS2NLBS3 which then pushes the Cursor back to the right column.

Once you add more than one character per row its all over anyways.

Wouldn't the backspace go back up to the previous line, since the only character between each of those numbers is the newline?

In there definition control signals position the cursor (typing head) on the screen (paper in typewriter), they don't operate on the command stream.

You are absolutely right that I didn't consider that. Thank you!

The reason is because I have just been typing into a table in a WYSIWYG editor which undoes the problem for me, returning to the beginning of each cell.So I tricked myself, because that is of course not what's being discussed in the article.

> Once you add more than one character per row its all over anyways.

Ah, but that can be solved by returning the carriage to the start of the line :)

Thus giving you the perfectly valid control sequence: LFCR!

Which can be easily replaced with a single control sequences since parts never make sense on their own

Typing in tables, etc. Is a reasonable use case. Especially on a line feed printer or typewriter from which these movements descended.

True, if you assume a table here has 1 column. If you are printing multiple columns you print a full row at a time.

The article doesn't mention possible security implications. However, we already get lots of vulnerabilities exactly _because_ implementations disagree on delimiters. Examples for this are HTTP request smuggling[1, 2, 3] and SMTP smuggling[4].

As the references show, this is already a big source of vulnerabilities - trying to push for a change in standards would likely make the situation much worse. At the very least, old unmaintained servers will not change their behavior.

I think we should accept that this ship has sailed and leave existing protocols alone. Mandate LF and disallow CRLF in new protocols, that's fine, but I don't think we should open this particular Pandora's Box.

[1] Simple example that doesn't use CRLF/LF disagreement: https://portswigger.net/web-security/request-smuggling

[2] Complex example that uses CRLF/LF disagreement: https://portswigger.net/web-security/request-smuggling/advan... (see heading 'Request smuggling via CRLF injection')

[3] Random report on HackerOne I found where allowing LF created a vulnerability in NodeJS: https://hackerone.com/reports/2001873

[4] https://sec-consult.com/blog/detail/smtp-smuggling-spoofing-...

The author makes this assertion:

> CR by itself is occasionally useful for when you want to overwrite a line of text you have just written. LF, on the other hand, is completely useless. Nobody ever wants to be in the middle of a line, then move down to the next line and continue writing in the next column from where you left off. No real-world program ever wants to do that.

Just for curiosity, isn't that what Whisper does, the data file format for Graphite? I remember reading about it in https://aosabook.org/en/v1/graphite.html

I realise this isn't core to the author's point, and I mean the question to learn something rather than trying to be a pedant. I think it uses python byte offset semantics rather than CR, but for all I know the python implementation uses CR under the hood.

The article overall is a fun read! I love the idea of trying to identify simple pieces of legacy ideas and clear them out.

>Once all the software has been fixed (except for SMTP, which is unfixable, in a multitude of ways even beyond its choice of line endings), we can all stop worrying over "\r\n" and just send simple "\n" line terminators.

IMO the day where we don't program computers anymore will come before the day everyone gets to forget what /r means.

Most software I make, makes the CR optional (although some disallows it), and I do not usually use it in my own files, except DOS files; I still use CRLF in DOS files, as well as in existing files that already use CRLF for some reason.

There may be some situations where it is helpful to have them separate, although for most uses (including most internet protocols) it isn't.

But, I agree with the part about Windows that O_TEXT should not be used; you should use O_BINARY.

I wrote a standalone CLI tool (written in Go) to determine and detect the end-of-line format, tabs, bom, and nul characters:

https://github.com/jftuga/chars

This is strangely written. NL is not a character. It is an operation. Every time he writes "NL character" he means an LF, i.e. 0x0A.

The NL operation is defined differently for different OSs. Yes, CRLF is a relic of the teletype, but in practice, it's really the Windows convention inherited from DOS. L/unix has always been LF. On old Mac operating systems it was CR alone.

> NL is not a character.

Unicode does in fact have a NEXT LINE character, 0x85. But it's not what the article is talking about.

Reply on news.ycombinator.com