GitHub satanically messing with Markdown – changes 666 to DCLXVI | Hacker News Reader

GitHub satanically messing with Markdown – changes 666 to DCLXVI

stackoverflow.com

Reddit also messes with numbers followed by periods at the start of lines. It doesn't convert them to roman numerals, but it does infer from their presence that you are trying to make a numbered list, and it apparently also assumes that you have no idea how to count, so it renumbers the list for you.

This regularly screws up posts in /r/chess. Someone might post something like this:

  1.e4 e5 2.f4 exf4 3.Nf3 g5 4.Bc4 g4
  5.O-O gxf3 6.Bxf7+ Kxf7 7.Qxf3 Bc5+
  8.Kh1 d6 9.c3 Nc6 10.d4 Nxd4 11.cxd4 Bxd4
  12.Bxf4 Nf6 13.Bg5 Rg8 14.Qh5+ Rg6

and Reddit changes it to this:

  1.e4 e5 2.f4 exf4 3.Nf3 g5 4.Bc4 g4
  2.O-O gxf3 6.Bxf7+ Kxf7 7.Qxf3 Bc5+
  3.Kh1 d6 9.c3 Nc6 10.d4 Nxd4 11.cxd4 Bxd4
  4.Bxf4 Nf6 13.Bg5 Rg8 14.Qh5+ Rg6

Reddit also uses commonmark, plus some GitHub flavored markdown extensions, via the comrak library, plus some custom extensions. So Reddit markdown should mostly look like GitHub.

Reddit seems to use two different Markdown engines. I use old.reddit.com, and I've noticed that some users that use new Reddit post code in ``` blocks. That doesn't work on old Reddit where you need to put four spaces in front of every line for it to render as code, so it just looks messy. I'm guessing their WYSIWYG editor uses the ``` version.

I only use old reddit and can post/view ```code blocks``` without issue.

I went looking for some examples and found these two. The first [0] is someone that tried to post a ``` block, but it's not rendering properly on old Reddit. It does render ok as a code block on new Reddit in a private window.

The second [1] is somebody that posted a ``` block but also indented all lines with four spaces. That renders as a code block with ``` lines before and after it on old Reddit, and as extra indented code on new Reddit.

If you use RES you can look at the raw Markdown code for each comment using the source link.

Maybe you´re thinking of inline code which use a single ` before and after the code content?

Screenshots: https://imgur.com/a/TKSKoEB

[0] https://www.reddit.com/r/cpp/comments/e16s1m/what_is_wrong_w...

[1] https://www.reddit.com/r/cpp/comments/e0fdp3/simple_timer_fo...

You're correct, I was thinking of `inline code`.

The old Reddit formatting wiki page doesn't mention ``` now I look, there's a note on new Reddit about sticking to spaces for compatability: https://www.reddit.com/wiki/markdown Would be nice if they fixed it indeed.

That's how Markdown is supposed to treat numbered lists. This section describes it in John Gruber's original description of Markdown syntax: https://daringfireball.net/projects/markdown/syntax#list

> It’s important to note that the actual numbers you use to mark the list have no effect on the HTML output Markdown produces.

It does cause a lot of surprising errors though, especially when the person isn't actually trying to write a list. For example, a case that comes up often is that someone will ask a question like, "when did this happen?" and someone will reply something like, "1986. It was a rough year." But due to markdown interpreting that as a list, it will get rendered as "1. It was a rough year."

Causes a lot of fun and confusion around ages too when it looks like someone's saying their child/significant other/etc. is one year old when they really typed a completely different age.

> That's how Markdown is supposed to treat numbered lists.

Huh. Is there a some reason why that's not obviously, bafflingly obtuse?

(Kind of like how Markdown and HN won't allow you to single-space text except in a code block)

It’s a really convenient property if you’re maintaining long lists since you can easily re-order items in a list without having to re-number each item.

Yeah, it's great for wikis. Less so for forums.

It seems like for those situations it would be much nicer to just have something like \item in LaTeX. It's still readable in plaintext, but wouldn't break things.

That's a perfectly reasonable use case that should be supported. But it should require explicit activation (a tag, etc). A formatting engine should not rewrite content unless told to explicitly.

From the section Deimorz linked:

> The point is, if you want to, you can use ordinal numbers in your ordered Markdown lists, so that the numbers in your source match the numbers in your published HTML. But if you want to be lazy, you don’t have to.

However, the complete disregard for the actual numbers seems to have been for ease of implementation in markdown.pl (the original implementation, which is what this doc is describing). The next paragraph notes:

> If you do use lazy list numbering, however, you should still start the list with the number 1. At some point in the future, Markdown may support starting ordered lists at an arbitrary number.

> starting ordered lists at an arbitrary number

The solution would need to be not just that, but also letting every li be arbitrarily enumerated. And at that point it's not even an ol, it's a ul (from a machine perspective) where the author might happen to mimick an ol. Or just nothing structured at all -- which would probably be a net positive.

The underlying HTML <ol> element allows arbitrary values through the value tag.

One of the original selling points of Markdown (at least in Gruber's post) is that is "seamlessly integrates" with HTML, in that you are in principle able to mix and match Markdown and HTML and get predictable HTML output. I think it was that aspect of Markdown that let it be so... sloppy?

That aspect of Markdown seems to have fallen away with the rise of Markdown extensions and standards.

By single space do you mean using a line break instead of a new paragraph? Markdown does allow for this: https://daringfireball.net/projects/markdown/syntax#p

The fact is that Markdown is not a very good design but became the standard. So this is just a cross one must bear.

> won't allow you to single-space text except in a code block

Why do you want to single-space your text though?

Because I want to present a list of items, or some reference URLs, or a poem, without tons of unnecessary white space.

Let's say you have a list with 50 items and want to add one item in between. Instead of updating all the indexes, you just type a number and Markdown does that for you.

In any case, it can be escaped by writing 1\. instead of 1.

I can't think of any reason I would want half my list indexes to be wrong in my source file.

I can think of plenty of reasons I would want to use auto-numbers while removing explicit numbers. But this is insane as a default behavior.

> I can't think of any reason I would want half my list indexes to be wrong in my source file.

Agreed. The bit about "oh, well, it fixes mistakes if you rearrange items" falls in the astonishing (in a bad way) camp far too often. When you really meant to number the lines, "2, 4, 6, 8" and it "helpfully" changes this to "1, 2, 3, 4". Arguably it's bad also in a list that has been maintained over time and has out-of-sequence numbers, where the users are counting on the processor to renumber - now, if someone says to change item 37, is that the one numbered 37 in the rendered output, or in the source? Better to use a tool of some sort to physically renumber that section of the source document.

A better alternative for auto-renumbering would be to only have it kick in when the entire list all starts with "1. " and there's more than one line in the list (so "...meet me at<next line>1. Then we'll..." doesn't end up with a line being dragged into being a numbered list). Then, the repeated use of "1. " at the start of the line becomes a much more obvious indication that you want to numbered list.

You can escape with \5.

The whole value proposition of markdown was supposed to be that a writer/reader need not create a detailed mental model to mentally parse the text. All of these are examples show why markdown (parsing/specification) is "broken" for use cases outside the straight and narrow. No big deal for now... Platforms which don't want to punish their users will accrete special cases and escape characters -- till some time in the future where the language will start feeling heavy with un-intuitive behavior. And so the cycle goes. :smh:

The point of the numbering changes is if you want to insert a new item half way through the list you don't have to change every single line. Unfortunately it probably confuses people more often than it helps them.

I'd also add that while I agree with the confusion expressed, using code snippets to escape from markdown parsing is important when sharing stuff that literally contains markdown. Of course that depends somewhat on the provider and what custom flavour of MD they use, which is often contraversial.

Like #463 GitHub auto linking, which I usually like, but occasionally been caught by it.

I've come to the conclusion that the worst thing about how developers design their products is the existence of "features" that override the user without the user ever telling it to. It assumes you wanted to do something other than was was inputted without getting further input from the user indicating they wanted to carry out that action. Pretty much every massive technology fuck-up in my life has been the result of the computer doing something other than what I told it to do and me not realizing or not being able to fix it after it did so. Things like "open restored document" helpfully deleting the older, unrestored document for me, attaching signatures to emails when I didn't configure a signature, etc. I want to start a movement that forces us to ask every new feature "did the user tell the computer to do X and if so, where is it indicated that X will happen?" There are a million examples when you look out for it and they're all infuriating.

Oh thank god I'm not alone in thinking that this is a horrible way to design a "feature"!

I've had very similar experiences as you. Every automatic "correction" of user input needs to at least be clearly indicated and easily reversible. In my opinion it shouldn't even happen until the user actively signed off on the proposed modification.

Ehhhh I agree with your general point, but I'd argue Markdown's list behavior shouldn't be tarred with this brush. It's a reasonable feature that makes perfect sense if you understand it, but in practice it didn't survive contact with the real world. Nothing is perfect, after all.

> It's a reasonable feature that makes perfect sense if you understand it, but in practice it didn't survive contact with the real world.

I think the same argument can be made for almost every "the machine knows better than the user" decision, so it seems to me that this is pointing out that the Markdown list behaviour should be tarred with exactly the same brush.

> It's a reasonable feature that makes perfect sense if you understand it,

This is true of all these features in general. They're all well-intentioned. No one wants to design a system that annoys users. But my criteria is meant to illustrate why it might not have been a smart idea to begin with: the user didn't indicate they wanted the system to renumber their bullet points. In fact, very often the user indicates that they want a specific numbering by placing specific numbers. The feature would work if there was some way to indicate this is just a bulleted list with numbers. Instead, the system assumes 100% of the time that the user made an error, then fixes that error in a way that is difficult to detect without ever providing feedback about the changes. That's a bad system that happens to usually be trivial or at most embarrassing. But the design itself is flawed from first principles.

A better way to implement auto-numbered lists would be indicate you want to use auto-numbered lists, just like you indicate you want to use headings, links, or any other element. First, don't rewrite all numbers, making the fact that a correction is taking place under the hood invisible to even experienced users. Auto-numbered lists should be indicated by having multiple lines starting with, say, "<space>0.", or even better "<space>#." to be more congruent with the use of "#" for headings and subheadings in other markdown systems (languages?). Second, when a user submits a post, the system should indicate what has been changed, perhaps with a non-intrusive highlight.

The problem would never have arisen had the developers asked my two questions "did the user indicate X?" and "where/how did the user indicate X?"

I suppose you should add onto that "is the change transparent to the user?" which is also known to cause some potentially life-threatening errors: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_... - you probably think "it's markdown, what's the big deal?", but who knows. Maybe someone gets some medical or chemistry advice and, sure, they shouldn't trust reddit, but then maybe someone dies anyway. So I think it's always best practice to indicate that a change has been made.

Hitting far above its weight in this category is autocorrect (my wife took to calling it autocarrot in a small act of rebellion), which "helpfully" corrects millions of painstakingly correctly-spelled obscure words into completely wrong but more common words, every day, making human communication less understandable.

Or there's no point and it's just a consequence of turning numbered lists into <ol>.

I am working in ATC (air traffic control). Controllers prefer a dumb tool with easy to predict deffects (like STCA) instead of a smart tool with infrequent random failures. Simplicity is underrated.

I take your point, though I’m less bothered about errors on Reddit formatting than I am errors in air traffic control.

Besides confusing people, I'd say it's a useless feature for a couple of reasons:

1. If you insert a misnumbered item in the middle of the list then when you come back later to edit the text you will be confused by how the numbers in the source are different from the numbers in the formatted output.

2. There's no point in using a numbered list unless you're going to refer to the number(s) somehow. (In this case I'm using the word "couple" to introduce the list!) Markdown doesn't sort out those references for you so in practice you have to manually renumber things anyway.

that's what plain html is for! I don't really see the point of markdown. With modern html5 you do not even need to close most tags.

IMO, plain HTML is much more confusing and quite a bit less legible.

I don't know. Even though I know HTML, I'd much rather create a bullet list by putting an asterisk in front of each item than writing a bunch of <ul><li> tags.

> I don't know. Even though I know HTML, I'd much rather create a bullet list by putting an asterisk in front of each item than writing a bunch of <ul><li> tags.

But I wouldn't—and it's a lot easier for you to process your Markdown and paste it into an HTML-enabled text field than it is for me to process my HTML and paste it into a text field that understands only one of the flavours of Markdown that doesn't have sufficient escape hatches built in (or that, like most flavours, has lots of features built in that you're apparently supposed to discover only by experimentation, preferably without any sandbox or preview available).

This is like dealing with that clueless newbie who desperately tries to help you, in the process creates a huge mess. You'd tell them to stop, and ask for permission before doing anything.

I propose a new term: paternalistic software. It's software that overdoes being helpful, and becomes annoying or dangerous. It's the software that does things you don't want automatically, because it thinks it knows better than you. Automatic renumbering and autocorrect both fall under this.

Sounds like my web browser.

I used to call it "overly-clever syndrome", but "paternalistic software" is way better. Thanks for this word.

> It's software that overdoes being helpful, and becomes annoying or dangerous.

It looks like you're trying to write a rant against Clippy. Would you like help?

Clippy was annoying, but at least it was personified enough to serve as a single point of blame. But more than that, it didn't actually do anything automatically for you. It was kind enough to ask.

I use this "feature". But generally I use the same number for all items because it is a pain to have to renumber everything.

maybe the solution is to renumber if a list uses all 1's, but leave it untouched if it has different out of order numbers.

So this gets renumbered:

1. item 1

1. item 2

1. item 3

But this does not:

Shouldn't things like numbering be part of the UA, then the user can decide (directly or give the job to a plugin).

Reddit could then offer "use our text-editor assistant in your browser" to those who aren't using one.

The text box would have to define what it allowed as input: alnum, plaintext, markdown, etc., and the appropriate plugin would activate?

I've got it. We'll create a _new_ spec and design it right this time. It'll support split ordered lists, and table colspan, image size attributes, and definitely templating - cause what we really wanna do is programmatically generate this stuff, amiright!?

I both hate and agree with what you wrote.

> The whole value proposition of markdown was supposed to be that a writer/reader need not create a detailed mental model to mentally parse the text.

The main value proposition was that the raw markdown looks like normal text. If you just want a simple mental parsing model, you use html or bbcode.

Addenda: looks like normal text --> looks and behaves like normal text.

If it looks like normal text but get rendered to a different text it is a negative value proposition.

Then just write plaintext. Markdown is not supposed to render exactly as written.

Most markdown parsers don’t work properly you begin with. The issue isn’t the text written. It’s poor parsers.

> Most markdown parsers don’t work properly you begin with. The issue isn’t the text written. It’s poor parsers.

I think this isn't exactly fair, in that it seems to mislocate the problem——Markdown itself is so underspecified (I think Gruber never really expected it to take off the way it did) that the meaning of 'Markdown' can vary wildly from place to place, as a result of which there's no simple, universal test suite, and it's hard to tell on what sort of edge (or not-so-edge) cases your homerolled parser will trip until you actually deploy it (at which point people have come to rely so much on its quirks that it's difficult to fix it).

Indeed, Gruber has resisted efforts to more thoroughly specify parts of the syntax (resolving ambiguities in parsing), saying that the original form works for his needs. This "power vacuum" has led to things like CommonMark (https://commonmark.org).

That is not the point; if markdown looks like text A and render like text B looking like text A is not a strength in and of itself.

I would say this would still fall under the realm of readability and of being lightweight (as in a markdown text looks like someone that did not knew markdown could have written)

But it is supposed to render exactly what was written. You write a heading, it gets rendered as a heading. You write the number 2, it ought to be rendered as the number 2.

I have no idea how this became the widely held belief about markdown. Markdown is based on Gruber's buggy Perl implementation of a rich text format that's intended to be fairly readable while editing the raw source files. Gruber had zero concerns for how other people would use it, he didn't even support standardisation efforts that tried to resolve the contradictory informal spec he had published.

To be fair, his use case was/is for writing very simple things (blurbs on his blog), that simply don't get into painful corner cases. It's a simple tool that works great - for simple things. It can fail hard on more strenuous cases. As you say, he resists attempts to resolve the ambiguities, yet, he also guards the name "Markdown" - which has all the mindshare - causing headaches for others who try to resolve those ambiguities. So, we get things like CommonMark.

I mean, they wouldn't have really wanted it to be formatted like a list, even if the numbering was correct.

Format it as code (indent with at least 4 spaces). Problem solved.

Problem worked around.

Backslash escaping the number (or possibly the period) should work in most (and therein lies Markdown's biggest problem, the lack of a proper spec (fingers crossed for CommonMark)) parsers.

That's some insidious auto-incorrecting there!

I actually hate all auto-"correction" features with a passion. After every Office update at work I have to go into the Outlook settings and manually disable all of that crap, but that still beats the hell out of having to live with it.

Have you asked your admin if it's possible to keep your Office settings through updates? I'm assuming you have, but you'd (possibly) be amazed at the stuff people don't mention to their IT guy...

Literally one client just "lived with" one of their screens being yellow. Never thought to mention it.

There's often a send that "this is what my employer have me to work with, I'm getting on with it". Perhaps there needs to be some effort to poll for issues rather than rely on interrupts.

Oh, if I even had an assigned administrator! Fifteen years ago, there was indeed an "in situ" administrator for the department. Five years ago, I could at least still find a couple of that ever farther removed breed of man in a room of the same building. Nowadays all we have is two alternative methods to have a "ticket" created, namely telephone and email. The main job of the first line of "support", located in a low-wage country, is to have every ticket you create closed as quickly as humanly possible. If your request is not on the (short) list of standardised tasks (like package assignment) it will usually get closed as "not an issue, won't fix". You then have to re-open and escalate, to get to someone who is capable of more than just following a few simple instructions from a process guideline. Even then, anything that is out of the norm will usually end in someone saying "this kind of request has not been anticipated" or "I'm probably not allowed to do that". Frankly, most of the time I don't even try anymore. Welcome to 1/2M employees company bureaucratic nightmare.

The "solution" is to make it "not a list" by escaping every first "." in your first example with a backslash.

Yeah, this one _is_ annoying. I'm not one to post chess games, but I've run into this one a bunch in other situations.

The _only_ case where this particular behavior is helpful is where one starts _every_ entry in a numbered list with (the same literal) "1. ", and lets Markdown sort it out - and that can be handy, because you can rearrange the list items and they still number properly, but in every other case it fails - it's a feature that produces more astonishment than happiness.

Kind of disappointing to see that level of testing failure. A leap to assume there is something specifically strange about `666` without even trying `665`...

Which link are you more likely to click on?

- GitHub satanically messing with Markdown - changes 666 to DCLXVI

- GitHub changes numbers to Roman numerals in lists

I’d call it functional clickbait.

In the CSS for github [1], there is this

  ol ol, ul ol {
    list-style-type: lower-roman;
  }

I think that will result in all items starting with numbers in lists using Roman numerals.

1. https://github.githubassets.com/assets/frameworks-481a47a969...

Can we have BBCode back? That was better.

Better for some things / worse for others.

It was much easier to write parsers for though. Most of the BBCode tags could be rendered with simple regex substitutions.

>It was much easier to write parsers for though. Most of the BBCode tags could be rendered with simple regex substitutions.

Nevermind. Let's not bring BBCode back.

On the whole I'd agree. When markdown is done right it's pretty awesome. However I'd still take BBcode over buggy half implemented markdown parsers like the one we have on HN. At least [i] is a less common token than * -- which leads to no end of edits when I've absent-mindedly emphasised something that should have been bullet pointed.

...but yes, I do agree that markdown is definitely better when it's done right.

On second thought, let us not go to Camelot. It is a silly place.

Seriously... https://blog.codinghorror.com/regular-expressions-now-you-ha.... And this piece of computer trivia archeology http://regex.info/blog/2006-09-15/247

> Seriously... https://blog.codinghorror.com/regular-expressions-now-you-ha... .

The key words I used were "most" and "simple". I'm well aware of the pitfalls of regex and have seen many horror stories myself but I'm not talking about parsing HTML nor other complex documents and nor am I suggesting you write the entire parser in regex. I'm talking about simple find and replaces for common static tags when you're only supporting a subset of BBcode. eg:

    form := "this is just a [b]crude[/b] example"
    rx := regexp.MustCompile(`\[b\](.*?)\[/b\]`)
    form = rx.ReplaceAllString(form, `<strong>$1</strong>`)
    fmt.Println(form)

(https://play.golang.org/p/GK7OGt2mHb6)

You could even use string find/replace but that wouldn't capture missing closing tags.

Things obviously get more complicated if you want to support some of the BBCode supersets like:

    click [url=example.com]here[/url] to load example.com

...but by that point you're dealing with a reasonably competent contributor who might feel better at home with markdown anyway (I know I would).

> And this piece of computer trivia archeology http://regex.info/blog/2006-09-15/247

Doesn't feel much like archaeology to me because I used to work a lot in Perl in the late 90s and 00s :) Though, and at risk of re-igniting old flamewars, I always preferred vi to emacs even despite having a fondness for LISP.

> Though, and at risk of re-igniting old flamewars, I always preferred vi to emacs even despite having a fondness for LISP.

That's because vi is a better text editor (as the old saw goes, emacs is a nice OS, but it lacks a decent text editor.)

I always found it very confusing to figure out exactly what tags it wanted since there was so much variation between forums.

I found the base set of tags didn't vary so much but there would often be a superset of features that would vary significantly as well as some sites that only offered a smaller subset of tags. However markdown is really no different in that regard.

Yeah until you denote your lists with #) and get to number 8 and now number 8) in your list is a really rad smiley. Everything sucks, let's go back to pen & paper ;)

So they change enumeration lists to Roman numerals. That's it, right?

Can get an (2017) on the title?

This seems like click bait to me - as if Github is censoring 666. But they aren't. It's just a really weird markdown bug having to do with the '.' which immediately follows the number. It's explained in the actual SO post.

It's a cool post but it has nothing to do with censorship. Am I the only one who was misled?

Neither HN nor SO are clickbaity. The user asking the question is saying there's something creepy happening, not that GitHub is censoring.

> which immediately follows the number. It's explained in the actual SO post.

Yeah, which is why it is on HN, what's your point?

Please don't respond rudely, even if another commenter is mistaken.

I think the title was just being light-hearted.

I don't think the two are mutually exclusive. The author of that SO question likely picked a light hearted title intentionally because it was "clickbaity".

It's a nice touch that the number of upvotes (09:10 Standard European Time)is also '666'

It should probably change it to χξϛ (https://en.wikipedia.org/wiki/Number_of_the_Beast)

Reply on news.ycombinator.com