mypyc is cool and all, but I can't help thinking about how Node just JITs everything automatically without the need for any special steps like this.
It just makes me sad that in a world with multiple high-performance JIT engines (including pypy, for Python itself), the standard Python version that most people use is an interpreter. I know it's largely due to compatibility reasons (C extensions being deeply intertwined with CPython's API).
There is a really important (if not "magic") difference between JavaScript and Python. JS has always (well, since IE added support) been a language with multiple widely-used implementations in the wild, which has prevented the emergence of a third-party package ecosystem which is heavily tied to one particular implementation. Python on the other hand is for a large proportion of the userbase considered CPython, with alternate implementations being second class citizens, despite some truly impressive efforts on the latter.
The fact that packages written in JS are not tied to (or at least work best with) a single implementation is also what made it possible for developers of JS engines to experiment with different implementation approaches, including JIT. While I'm not intimately familiar with writing native extension modules for Node (having dabbled only a little), my understanding is the API surface is much narrower than Python, allowing for changes in the engine that avoid breaking APIs. But there is less need for native modules in JS, because of the presence of JIT in all major engines.
this is misleading, if one sees the phrase "interpreter" as that code is represented as syntax-derived trees or other datastructures which are then traversed at runtime to produce results - someone correct me if I'm wrong but this would apply to well known interpreted languages like Perl 5. cPython is a bytecode interpreter, not conceptually unlike the Java VM before JITs were added. It just happens to compile scripts to bytecode on the fly.
Almost all execution techniques include some combination of compilation and interpretation. Even some ASTs include aspects of transformation to construct them from the source code, which we could call a compiler. Native compilers sometimes have to interpret metadata to do things like roll forward for deoptimisation.
But most people in the field would describe CPython firmly as an 'interpreter'.
Yeah? I think almost everyone would?
> and you'd call Java an interpreted language?
Java is interpreted in many ways, and compiled in many ways, as I said it's complicated. It's compiled to bytecode, which is interpreted until it's time to be compiled... at which point it's abstract interpreted to a graph, which is compiled to machine code, until it needs to deoptimise at which point the metadata from the graph is interpreted again, allowing it to jump back into the original interpreter.
But if it didn't have the JIT it'd always be an interpreter running.
This is a common implementation approach - parse the source to generate an AST, transform the AST to bytecode, then interpret the bytecode. It's still interpretation, and is slow. Contrast to JIT engines which transform the intermediate code (whether that's AST or bytecode) to machine code, and is fast.
Perl uses the same execution method you describe for cPython.
(Not trying to interrogate you or prove you wrong, but I've got an interest in optimising very difficult meta-programming patterns.)
This means nothing is guaranteed and so every instruction must do multiple checks to make sure data structures are what is expected at the current moment.
This is true of JS as well, but to a lesser extent.
Aren't all the things you mentioned already fixed by deoptimisation?
You assume constants cannot be modified, and then get the code that wants to modify constants to do the work of stopping everyone who is assuming a constant value, and modify them that they need to pick up the new value?
> To deoptimize means to jump from more optimised code to less optimized code. In practice that usually means to jump from just-in-time compiled machine code back into an interpreter. If we can do this at any point, and if we can perfectly restore the entire state of the interpreter, then we can start to throw away those checks in our optimized code, and instead we can deoptimize when the check would fail.
https://chrisseaton.com/truffleruby/deoptimizing/
I work on a compiler for Ruby, and mutable constants and the ability to monkey patch etc adds literally zero extra checks to optimised code.
You can write a new compiler if you'd like, as detailed on this page. But CPython doesn't work that way and 99% of the ecosystem is targeted there.
There is some work on making more assumptions as it runs, now that the project has funding. This is about where my off-top-of-head knowledge ends however so someone else will want to chime in here. The HN search probably has a few blog posts and discussions as well.
Yeah that’s the point - the JIT takes that capitalisation as a hint to treat it as a true constant and bake the value in until it’s redefined.
This is all solved stuff and isn’t a barrier to implementing a powerful JIT for Python if someone wanted to.
I'll give you one you could have used - the GIL - however I'm not sure the GIL's semantics are really specified for Python, they're an implementation detail people accidentally have relied on.