Hacker News Reader

Comment by chrisseaton | Hacker News Reader

Ok but those are standard, valid characters like any others. Why are they ‘horrid’?

Some people hate unicode quotes because lots of software will automatically convert ASCII quotes to them when you copy/paste your code to Slack or email to share with others.

Because they result in a syntax error.

But they wouldn’t do what you wanted anyway if they were ASCII. What’s the problem specifically with them being Unicode?

The problem specifically is that they are not the ASCII quote character. There is only one ASCII quote character, and that's the one used by programming languages. Any other quote or quote-like character is outside the ASCII range, and must therefore be Unicode (or another non-ASCII code page).

I know they're not compatible with programming languages that you may want to use. But that's the languages' problem. They're perfectly valid, standardised, characters.

No, iBooks inserting them around commands is the problem.

Inserting ASCII quotes instead would result in exactly the same problem.

Not exactly the same. Since anything inside ASCII quotes is typically considered a valid string in most programming languages, if you delete everything after the final quotation mark you have a syntactically valid program. It won't do what you want, but it's not a syntax error.

So unicode quotes and ASCII quotes are not exactly the same in this scenario.

They both break what you were trying to do, and the fact that they do it in slightly different ways really isn't worth making a long pedantic argument about.

Programming is literally the art of pedantically telling a computer what to do, arguments about it are naturally going to tend towards pedantry. So that's my first point.

The second is that there's a significant difference between "won't run due to syntax errors" and spitting a pasted line back at you since it thinks it's just a string. In a less robust environment the first option might actually crash your environment, or leave you with subtle errors. Like in a shell, it might treat things like escape sequences and things will look off unless you reset them.

Even though the OP seems offended by the presence of the Unicode quote marks for some reason, they have nothing to do with the issue of Apple Books surrounding the copied text with those quote marks and adding an attribution line. If they'd surrounded the text with ASCII quote marks and added an attribution line, it would have still been a syntax error, wouldn't it.

Personally, I don't see any reason to hate typographically correct quote marks when used correctly, which (obviously) doesn't include code samples.

They're horrid because you can't easily type them on a keyboard.

Because for some reason terminals are stuck in the 70ies and don't accept those characters as quotes. Anything but ASCII trips them up.

Seems such an obvious interface to innovate, but it seems to run into terminal wizards sense of purity.

The problem, such as it is, is with languages. Terminals (mine at least) handle most of Unicode just fine; admittedly I've seen it choke on emoji, but punctuation, nah.

The vast majority of programming languages are defined in terms of ASCII and only ASCII. I don't care for this, personally.

I've given some thought to how to do quoting right in a programming language, and implemented «guillemets» as an experiment. But it's challenging, you need to decide what to do with all of “”‟„"″ and there aren't obvious pairings, like „this is a sentence” and “this is a sentence” and »this is a sentence» and «this is a sentence» and »this is a sentence«, it ends up feeling like rather a lot of effort for what you get in return.

Oh, one of those characters I typed isn't a quotation mark, did you catch which one? Hacker news won't even let me type two of them!

A language or interface is defined by the set of symbols mutually agreed upon. If you allow Unicode that number simply explodes, thus bloating and complicating the language/interface and its implementation as well. In effect it is no longer the same language but becomes a different, more complex one.

The tradeoff is not worth it for programming languages the same way as learning all the scripts of the world is not worth it for one human being, just to accomplish tasks that don't need all these symbols in the first place.

> I've given some thought to how to do quoting right in a programming language

Already fully designed and implemented in Raku: https://docs.raku.org/language/unicode_entry#Smart_quotes

Test online: https://tio.run/##K0gtyjH7//9Rw7ySjMxiBSBKVChOzStJzUtOfdQw9/...

It is a solved problem however. We chose not tot use it (in terminals).

Having multiple almost identical ways of achieving things is a bug magnet in programming languages. Differing programmers will presumably use different styles (why else even support differences?), and if you mix code like that, bugs ensue. This a hassle not just because autoformatters are liable to make churny changes (which distract from real changes, which makes history harder to understand: bug magnet), but also because people will make mistakes e.g. when find-replacing (another minor bug magnet!). Then there's the fact that some of those quotes aren't symmetric - so you need to think of something to have happen when they're unpaired or used incorrectly, and it wouldn't surprise me if no matter what you did, you surprise somebody (bug magnet!).

Sure: these are all quibbles, and a language wouldn't die from all these minor cuts. But they're definitely downsides, not upsides. So: where is that upside? Why would you ever support something like this? "It looks a little nicer" sounds like a pretty weak argument compared to "it's inconsistent, hard to machine process, and may cause a few bugs"...

Many programming languages, certainly not all, offer single and double-quoted strings already. Which is, granted, a perennial source of annoyance for those of us trying to have a consistent formatting style.

I want «guillemet strings» because they use a matched pair, so you can «quote a string «within a string» without escaping» and I think that's a nice property.

I'm not really interested in supporting all the forms in which they're used in European languages, though, that would be a real hassle. Just the one that looks like the other matched pairs we use in programming.

“smart quotes” have ‟at least two styles” and really „three styles”, and the first two are really easy to confuse with "normal double-quotes". I don't want my users to have to deal with "why doesn't this compile”?, and if I allowed it to compile, now you have to escape all the quote characters inside any string, which is messy.

Raku, as a sibling points out, has bitten down on this bullet, and I respect that. I keep meaning to give it a spin, I'm fond of Parsing Expression Grammars and have good feelings about Perl from the early days.

PostScript has something like the «guillemet strings», but it uses parentheses instead. The parentheses can be nested without escaping.

Those work in Raku, with the exception of ‟at least two styles”.

    .say for q[‟”].uninames
    # DOUBLE HIGH-REVERSED-9 QUOTATION MARK
    # RIGHT DOUBLE QUOTATION MARK

If you use one type of quote, you can use the others inside of it with no problem.

    “double "quotes"”

(Raku doesn't use a tokenizer.)

Also note that you can use a variety of characters for quoting if you use `q`, `qq` or `Q` etc.

    q<<<<1 < 2>>>>  eq  '1 < 2'  eq  q^1 < 2^  eq  q%1 < 2%  eq  q｢1 < 2｣

Note that many Unicode characters which have a LEFT and RIGHT variant will also be paired when you do that. q⦑like this⦒

In Raku string literals are actually a [parameterized domain specific language](https://docs.raku.org/language/quoting). (The domain of creating strings.)

    my $buffer = Blob.new(115,116,114,105,110,103); # 'string'.encode()

    Q            # start with raw quoting
    :scalar      # enable embedding the value of scalars
    :backslash   # enable backslashing characters
    [This is a $buffer.decode()\n]

There are shortcuts for common uses

    'single'  eq  q [single]  eq  Q :single [single]  eq  Q :q [single]
    ‘single’  eq  ‚single’

    "double"  eq  qq [double]  eq  Q :double [double]  eq  Q :qq [double]
    “double”  eq  „double”

    ｢raw｣   eq   Q [raw]

    < words >   eqv   qw [ words ]   eqv   q :words [ words ]

    << quote words >>   eqv   qqww [ quote words ]   eqv   Q :double :quotewords [ quote words ]
    « quote words »

This is not exhaustive.

:double is actually short for :scalar :array :hash :function :closure :backslash

:words splits on whitespace :quotewords splits on whitespace but respects quoted sections

    q      :words [ a "b c" ]   eqv   ( 'a', '"b', 'c"' )
    q :quotewords [ a "b c" ]   eqv   ( 'a', 'b c' )

Note that :double is the same as any adverb in the language, so you can negate a feature with :!double.

    qq [with newline\n]
    qq :!backslash [without newline\n]

    qq :!array [user@example.com()]

(It's actually rather difficult to accidently use @ or % variables.)

That seems like an obviously bad solution. That is like writing [a(b{c]d}e) except that the quote characters look even more similar.

Its easy (solved) to map Unicode to control characters such as quotes (and when you think about it, quotes are often there to deal with ambiguities stemming from the limited number of allowed characters in ASCII). So to could have a terminal accepting such input, and a few helper function that normalize it into ASCII and so on.

After all, users of non-ascii languages (which is nearly everyone) already know how to deal with it without ambiguity. Its only ambiguous if you don't use encodings, and that should never happen anyway.

> and a few helper function that normalize it into ASCII and so on

If you are going to get rid of it anyways, why bother with it in the first place? Also, you then have the ambiguity/problem whether some symbols are mapped to the same ones or not.

> already know how to deal with it without ambiguity

I wish. Just dealing with ü vs ue vs \"u and ascii, win something, utf8 and utf16 is still annoying in practice. Or latex choking on accents, non-breaking whitespace or similar fun stuff.

The argument to do it is it democratizes users with their scripts around the world (and solve the issue of having 'the wrong' quote). Normalization solves the latter. It's really solved, but you would never know if you live exclusively on the command line. Which I think is unfortunate.

I disagree, it simply tells everyone they are wrong and the "normalized" one is the right one.

Oh, you used foo and föö as variable names, too bad those are the same now by normalization.

> It's really solved, but you would never know if you live exclusively on the command line. Which I think is unfortunate.

Tone down the condescension and insults, please.

The terminal can usually handle it, provided it's UTF-8. Most of the editors can handle it. It's very specifically the programming language specifications which don't include them. Even the ones that allow Unicode identifiers.

Also, they're difficult to type - my keyboard has a " key but not a “ or ” character. I had to copy and paste those from the unicode website since I don't have a suitable input method set up.

The idea that "terminal wizards" reject this is rather undermined by the fact that "terminal wizards" made open and close double quotation marks accessible as simply [Group2]+[B04] and [Group2]+[B05] on an ISO 9995-3 conformant keyboard.

Or [Shift]+[Option] [B04] and [Shift]+[Option] [B05] if there's no explicit [Group2] shift.

Open and close single quotation marks use the same keys.

"terminal wizards" have done quite the opposite of rejecting this.

Reply on news.ycombinator.com