Hacker News Reader

Comment by mehrdadn | Hacker News Reader

Python 3.7 to 3.8 broke some stuff. I was pretty annoyed to discover (the hard way) that e.g. this wouldn't run anymore:

  python3 -c "import ast; print(ast.arguments([]).args)"

It seemed completely unnecessary too. They could've just kept the structure the same across minor versions...

View on HN

I think that probably lesson learned from python 2 -> 3 migration. It's probably better to introduce many small breaks than a big one?

The python despite having 3 digit versions apparently is not following semantic versioning. The major version number generally had not much significance (the python 3 was exception, 1->2 and what they are assuring us 3->4 won't be a big change)

"Better" depends on your agenda. If it's to shove the next major version down users' throats, maybe it's easier to swallow small bites (bytes?) than one giant cow. If it's to provide a predictable and pleasant user experience, I'm less confident this is the right approach.

So I did look it up, they do provide a warning though (in the first paragraph):

"The ast module helps Python applications to process trees of the Python abstract syntax grammar. The abstract syntax itself might change with each Python release; this module helps to find out programmatically what the current grammar looks like."

This module is a bit special, because as they say, it supposed to reflect current python grammar. If they change grammar and didn't make it reflect it it would lead to different kinds of issues.

It's one thing to extend the AST and add a couple children because you added new syntax or something. That's what I think most sane people would expect from reading that quote as a language evolves. So yeah, don't assume a node has 4 children when it might have 5+ tomorrow. But it's another thing to outright remove a child entirely and to be so vague about it too. (They didn't even have it be a sentinel value like None, or even care to leave a useful error message in the stack trace! The message makes you think the code was broken all along and simply never triggered before.)

Lest you think informing users more clearly is a foreign concept to them, look at the dis module [1] for comparison. They're extremely clear the whole thing is implementation detail of CPython. If anything, after reading that, one would think your conclusion would be "ah, I should program against the AST then, not the bytecode". So you do that and then you're greeted with this nonsense! Obviously it's your fault for assuming there's anything stable to program against across minor releases.

[1] https://docs.python.org/3/library/dis.html

the child wasn't removed, it was renamed from arguments to posonlyargs, this is related to the change of allowing functions to specify that given arguments are always positional.

This module exposes internals of python, and providing such guarantees would cripple development, because it wouldn't even allow for refactoring the code.

Most languages don't run into this because they don't expose internals like that. You typically extract that yourself and you accept it can change every release.

As for dis, that's very different, the bytecode is just an optimization of the CPython, python code could work perfectly fine without it, the bytecode was introduced to make it faster and other implementation won't use I would imagine that Jython and IronPython most likely don't implement it since they have their own (JVM and .NET)

ast on the other hand is expected to be identical if a language claims to be compatible with a given version, and you would expect PyPy for example to also provide this package.

That is what is great about having a good test setup. You would know pretty quick if stuff broke. Which has been basically nothing for me (3.6 -> 3.7 -> 3.8)

To be clear, the pain here was hunting down the root cause and fixing it when it shouldn't have broken in the first place, not the speed of discovering the problem after the code had already broken.

Reply on news.ycombinator.com