Hacker News Reader

Comment by CameronNemo | Hacker News Reader

Simpler example of the issue:

    >>> 1or 2
    1

View on HN

Looks like the parser knows numbers end when the digits end so splits them into two tokens. (Ignoring the optional [lL] long suffix before Python 3)

One of those things that looks helpful at first glance, but is problematic in the long run. Throwing a syntax error immediately would lead to more robust/maintainable code.

Though you don't want to throw a syntax error on "1+2"

Yes, the difference between a letter character and punctuation/operator character.

There's an argument to be made that disallowing it could help in the long run as well, but Python has never been so strict. So I'd stop short of recommending it here. Perhaps in a bondage and discipline language. :-D

1+2 is a valid expression consisting of 3 tokens, "1", "+", and "2".

In most languages, numeric literals, identifiers, and keywords cannot be adjacent, but any of them can be adjacent to an operator. The odd thing here is "0xfor" being tokenized as "0xf" and "or". In C, for example, "0xfor" is an invalid token.

> In most languages, numeric literals, identifiers, and keywords cannot be adjacent, but any of them can be adjacent to an operator. The odd thing here is "0xfor" being tokenized as "0xf" and "or"

0xf is a numeric literal, and or is an operator, so as you said, they can be adjacent. or isn't an operator in C.

of course, if it were 0xfand ... you have a fun problem, because it could be 0xfa nd, but nd isn't an operator (that I know of), or 0xf and. Ideally, syntax shouldn't be ambiguous.

I should have stated that more clearly. In Python, "or" is both an operator and a keyword.

In most languages, numeric literals, identifiers, and keywords cannot be adjacent, but any of then can be adjacent to a token that consists of punctuation characters. (In some languages, most or all operators spelled using punctuation characters, which is what I had in mind when I wrote my previous comment.)

or is an operator in C++ and if you #include <iso646.h> in C

> or is an operator in C++

Yes.

> and if you #include <iso646.h> in C

Sort of. As a preprocessor token, it's an identifier that happens to be the name of a macro. As a token (after preprocessing), it becomes the "||" operator.

My preference would be that the syntax requires spaces in that case. Oh well, I've always got the option of writing my own programming language.

You have to recognize the specific case to throw the syntax error. Sounds like they're going to fix that.

I guess people don't remember PL/I very much: PL/I keywords are not reserved words, so it is possible to use them in a program in other than their keyword context. DO DO=1 TO TO BY BY;END=END+1;END; is a valid program.

For those wondering, the ideas behind that choice is that you can’t expect anybody to know all keywords. PL/I had many built-in functions that all were keywords, so it had many keywords (https://www.cs.vu.nl/grammarware/browsable/os-pli-v2r3/#Keyw... lists hundreds)

and even if you do, future versions of the language may include more keywords.

There's the additional thing with x not being defined, not not being evaluated, which I found surprising at least. So perhaps:

    >>> 1or x  # without x existing
    1

Clearly a lexer error.

Why is it "clearly" an error? This syntax has been around forever and is widely used by code golfers, so if it were unintentional, I'm sure the Python devs would've fixed it by now.

It no longer matters if it was originally unintentional. If it's in active use now, so even if was originally by accident it cannot be fixed without breaking backwards compatibility.

Eh, python breaks backwards compatibility between versions all the time. And I don't know if "it's used by codegolfers" is a legitimate reason to keep a language feature.

That said, I have a really hard time imagining this issue coming up in regular usage of the language so I don't really have much of an opinion should they choose to keep or drop it.

Reply on news.ycombinator.com