Shouldn't this be considered a bug in Python? Why does it even try to evaluate 0xfor without the space? Trying a few other things..
* 0xfor1 evaluates.
* 1or 2 evaluates.
* 1or2 doesn't.
* ''or'foo' evaluates.
This is gross.
* 0xfor1 evaluates.
* 1or 2 evaluates.
* 1or2 doesn't.
* ''or'foo' evaluates.
This is gross.
“1or2” is lexed into “1” (integer) followed by “or2” (identifier), which is valid on the lexer level but then fails on the grammar level.
I guess it’s an operator token after all.
https://docs.python.org/3/reference/lexical_analysis.html#wh...
This is not a bug.
> Whitespace is needed between two tokens only if their concatenation could otherwise be interpreted as a different token (e.g., ab is one token, but a b is two tokens).
The two tokens in this case are "0xf" and "or". Their concatenation cannot be interpreted as a different token, because "0xfor" is not a valid token. Therefore, if I'm reading the rule correctly, whitespace is needed in this case.
"0xffor" is another interesting case. It's also not a valid token, but it could be interpreted as two tokens in two different ways: "0xf" "for" or "0xff" "or". (Python does the latter. I presume it uses something like C's "maximal munch" rule.)
> Whitespace is needed between two tokens only if their concatenation could otherwise be interpreted as a different token (e.g., ab is one token, but a b is two tokens).
Because the concatenation of "0xf" and "or" can't be interpreted as a different token, the whitespace is not needed.
I dislike the rule, and I strongly think that "0xfor" should require whitespace between "0xf" and "or" (I'm sure that influenced my reading), but you're right about what the rule says.
(Apparently I can't edit my previous comment.)
But requiring whitespace between all tokens is not an acceptable solution, since "2+2" should work. Always equiring whitespace between alphanumerical characters in different tokens would make sense.