When a Space Is Not a Space | Hacker News Reader

When a Space Is Not a Space

bigmessowires.com

19 pointserickhill18 comments

It isn't just the usual non-breaking space. I've seen other space types like "En Space" creep into content copied into HTML entry forms, and cause issues further down the line (usually a ? appearing when an app tries to convert to a non-unicode encoding and doesn't understand these characters).

There are a few spaces in the Unicode standard: https://www.compart.com/en/unicode/category/Zs

And the list linked above doesn't include things like zero-width-joiner. See https://en.wikipedia.org/wiki/Whitespace_character for a fuller list.

If you want to be evil to a programmer, as well as being tricksy with white-space, replace some semi colon characters (U+003B - ;) with Greek question marks (U+037E - ;) and see ho wlong it takes them to work out why their compiler or linter isn't happy…

The takeaway from all this is that you should never assume plain text is simple.

Yeah I’ve encountered some really fun bugs on more than one occasion due to zero-width spaces sneaking into places they shouldn’t be. I hate Word documents as a medium for sending around API instructions or content for a CMS for this reason.

And then there are those curly quotes. shakes fist

In fact, every entity sequence (&...;) in HTML encodes a character from Unicode (one or more codepoints). So, for example,   refers to the Unicode character U+00A0 NO-BREAK SPACE. (Note that the bytes C2 A0 are simply the UTF-8 encoding of this character).

This is DEFINITELY a feature.

I bet users are copying and pasting values from the WordPress editor into a word processor and HATE seeing   so they probably asked for this solution.

html-based tools and/or source views failed

These do not have any control on how your text is copied, because your browser does that (it was the reason for an entire issue in the first place). Maybe they could convert A0 to   but then you couldn’t copy that text into non-html environments easily. It is inevitable for &lt/gt/amp;, but should they also convert e.g. dashes and other html-safe entities?

Putting Non-breaking-space-as-Unicode into HTML source is just a bad idea, because you cannot see that it's the non-breaking variant. Using ` ` instead would have been much better, because it's explicit and visible.

If I had to spitball, I'd guess that the issue was caused by the interplay between HTML's whitespace collapse algorithm and whatever method WP uses for source editing. That something is converting whitespace characters to U+0020 for collapse and display, but the conversion breaks copy-and-paste. Whitespace issues in HTML can get quite annoying at times.

Non-breaking space is very important for languages like Arabic and Farsi. Changing a non-breaking space to a normal space in some sequence of characters, may change the meaning of the word in these language.

“I’m not sure what happened, but I blame WordPress.”

A decent heuristic.

"This is like the 21st century version of confusing a zero with a capital letter O, yet worse."

I like the analogy, but why is it worse?

The confusion between between a zero and an O would only happen to a human. The software would always consider them fully separate characters, would never, for example, silently convert all the zeros to O's during a copy-paste operation.

Also, zero-O confusion can be remedied by simply choosing the right font. But by definition, different types of spaces all have to look exactly the same when rendered as text. To inspect and diagnose the problem, you have to switch to a different form of character representation, like hex or Unicode code points.

> But by definition, different types of spaces all have to look exactly the same when rendered as text.

Only in the sense that they look like nothing at all. They can, however differ in how much nothingness they represent [1] which I would argue can make them look quite different one another.

[1] https://en.wikipedia.org/wiki/Whitespace_character#Unicode (Feel free to ignore tabs, line breaks and other characters that don't feel space-y enough. There should still be plenty left to choose from.)

> The confusion between between a zero and an O would only happen to a human

It happens to a computer doing optical character recognition (OCR) on a scanned document. Or a machine vision system reading street signs or other text.

> But by definition, different types of spaces all have to look exactly the same

The article is in fact about a case where this didn't happen, and in general it seems weird to insist that different spaces must look the same.

Except for ` `, where the point is to look the same but behave differently.

Nice article!

Reply on news.ycombinator.com