Hacker News Reader

Comment by geofft | Hacker News Reader

At first glance, I'd consider it a (security) bug in Perl 6 that it permits tokens containing invisible characters, let alone consisting solely of invisible characters. Are there any other languages with this behavior?

As a random example:

    titan:~ geofft$ python3 -c "$(printf "\u2063") = 1"
      File "<string>", line 1
         = 1
        ^
    SyntaxError: invalid character in identifier

If you change it to e.g. 00e9 ("é"), Python 3 permits the character, so it's not just a lower-ASCII thing.

View on HN

https://rt.perl.org/Public/Bug/Display.html?id=128159 was filed by the author of this piece. The take-home to me is that supporting Unicode to a great depth in a language is really hard.

My company's large Java codebase has hundreds of zero-width spaces in the middle of method names. It's sad.

What's the reason? Were the method names needed to be aligned somehow?

"invisible" is entirely dependent on your text editor

What about carriage return. newline,form feed, etc? Those are invisible character, and those are just plain ASCII.

Those aren't invisible, merely transparent: they take up space, and alter the location of the cursor when inserted. Invisible characters don't.

Also, some text-displaying programs will insert glyphs for them, to make them visible, which would make them somewhat more detectable.

And the BEL character, while non-spacing and invisible, is sometimes audible.

Those aren't valid tokens. "<newline> = 10" won't assign to a variable named <newline>. If the language wants to parse non-ASCII invisible characters as whitespace, or permit them inside comments or strings, that's fine.

Reply on news.ycombinator.com