Hacker News Reader

Comment by jstimpfle | Hacker News Reader

Yes. I already tried to explain with an analogy about faithfulness (which I removed because it got many downvotes).

When you deal with ASCII, that's also the semantic level on which you operate. Much simpler than Unicode (I don't think there are many people who really understand Unicode).

When you have a Unicode string that uses only characters that are also present in ASCII, that doesn't make it an ASCII string. It has more meaning.

Btw that has nothing to do with BOM. BOM is itself not part of ASCII, and not encoded as a single byte, and it doesn't affect single-byte sequences.

View on HN

Are you talking about how, when someone is limited to ASCII, he makes substitutions he wouldn't make if he were using Unicode? Such as using :-) instead of U+263A?

Or are you maybe talking at a software level, where you can make some simplifications and assumptions about a string when it is guaranteed to be ASCII?

Because otherwise I can't see how there could be any semantic or meaningful difference between ASCII and Unicode when the text is the same. An ASCII 'a' has exactly the same meaning as a Unicode 'Latin Small Letter A'.

You can look at it from an information theoretic standpoint. The information you get from the string is also all the things that it does not contain.

Btw. I have no clue why I am constantly downvoted. Whoever did this, I don't think there is an incentive to downvote my two ancestor comments. It's demotivating.

Didn't downvote but I can see why they got downvoted: they make a claim, but when asked for a specific argument, they go roundabout.

In what SPECIFIC way is an only-ascii string not a UTF-8 string for ALL intends and purposes?

You throw some words like "semantics" but no actual arguments.

I think I answered that in the clearest way I (personally) could, right in the comment to which you replied.

But wherein does that meaning lie? If the two strings represent the same text, you can't tell the difference between them, because there is none.

(I guess technically you can't tell the difference between an ASCII string and a string of random 8-bit numbers in the range 0 <= x <= 127, but if when represented as ASCII characters those numbers are meaningful -- comprehensible English, for example -- the odds are good it's the former not the latter.)

I think what he's getting at is that whatever program is reading the text must be prepared to read UTF-8 and not just ASCII. Sure, the underlying data is the same, but the data type is not the same; analogously, a byte with the decimal value of 65 might be the number 65 if cast as an integer, but might be the character 'A' if cast as a char.

Reply on news.ycombinator.com