Pyjion – A Python JIT Compiler

I don't think that's really true. The Java language specification is entirely prose. The book you linked to was written by "outsider" authors and published in 1999 (!).

Can you point me at the proof for the soundness of the documented behavior java or javascript or C++ docs? A cute little table isn't a substitute for soundness, and none of the languages you mentioned are mores soundly implemented (at least in their popular implementations).

> It's easy to see where these simple English descriptions aren't covering everything. To give you a practical example - look at https://docs.python.org/3/reference/datamodel.html#object.__... - 'should return False or True' - what if it doesn't? Where's that specified? Is it somewhere else in this document? That's the kind of practical issue we work with when implementing languages.

Cpython raises an exception, and in general, cpython is the spec unless otherwise specified is how things turn out.

> How can you test against English prose? You can't. So someone's manually translated the prose into tests elsewhere I guess. Have they done that correctly? How can we verify that? Was there any ambiguity when they were interpreting the English?

This is sort of a silly complaint. every spec is implemented in english prose[1]. That's why we end up with arguments about SHALL vs. MUST in the specs. Except in the rare cases where the spec is a test suite, which usually reduces to the case of cpython: the popular implementation is the spec (or maybe the popular implementation forks its test suite out into a different repo to make it more "independent")

[1]: Please don't make a irrelevant point about an obscure implementation of C that's implemented in agda and the "spec" is the proof of soundness or whatever, that's fundamentally the same as the implementation is the spec, especially given that said C implementation probably isn't ANSI compliant or whatnot.

Besides the nuances of how everything is executed, there are occasionally breaking changes between minor language revisions.

You cannot just pick up a random Python script and be sure it still behaves the same way across all minor revisions.

I think the GP means character encodings.

So the intent here is to ask whether (for example) a foo defined in a ISO 8859-1 encoded file is overidden by a foo imported from a Windows-1256 encoded file.

1. RPython + the PyPy _compiler_ which is a compiler for JIT compilers (like GraalVM as I understand it)

2. An implementation of the Python language _using_ RPython to produce a JIT for Python scripts.

There are other languages _using_ RPython + PyPy compiler to produce JIT compilers for languages other than Python too.

https://doc.pypy.org/en/latest/architecture.html#layers

The extremly cool and awesome thing about this is that this is effectively a general purpsoe JIT, one JIT to rule all interpreted languages that could ever be written. There is nothing specific about Python in the toolchain. For _Any_ interpreted language:

- You write only your naive-but-readable interpreter in Rpython, a restricted subset of python that tries to preserve the readability but ditch the dynamic madness. (This is not python, this is an entirely different language. It just happens that every valid Rpython program is also a valid Python program. There is nothing special about Rpython here either, they could have theoretically picked any readable language to write your naive interpreter in, but they chose Rpython)

- The compilation pipeline produces two things: an exectuable image of your naive interpreter*, and a bytecode image for the general-purpose JITer.

- Normally, it's the executable image of your interpreter that runs your language's programs, but once it detects a user-program-level loop (e.g. because it has encountered a backward jump.), it invokes the supporting runtime (the general-purpose JITer) and delegates to the bytecode version of itself.

- The GP JITer starts tracing the bytecode image of your interpreter (which, remember, is itself executing the user-level program the whole time), once it detects that the user-level loop is done, it says so. Now the general-purpsoe JITer has a record of all the operations that your interpreter executed while it was running the user-level path, which is the same as {all the operations that the user-level path executed} (minus all the interpreter-specific operations, which the GP JITer also knows about because this info is contained in the bytecode)

- The GP JITer treats the execution record as any other JIT, it produces an optimised native version from it, and bingo!, you got yourself a native image of that user-level loop.

- The original interpreter, the executable, now goes back into the picture. It puts that native version of the loop in its pocket, ready for the next time it encounteres the loop.

It's so f*ing cool, that's why their logo is a snake eating itself: there's so much meta shenanigans going on. Their implementation of Python is merely the application, it's the amazing toolchain they built to build it that is the real treasure.

*: One of the steps in creating the exectuable is, I kid you not, is running the standard Cpython interpreter on your Rpython source (as it's valid python), waiting for interpreter to do it's expensive startup, then freezing the whole enviroment it produced to package it with the executable. This couldn't be done to speed up normal Python because its extremly dynamic nature messes with this.*

This gist talks specifically about PyPy using the Futamara projection.

I wasn't criticizing Pyjion, I was responding to coldtea's comment saying that people who can't run this in production would likely not be interested in trying it. I have zero issue with Pyjion, and I'm also happy with my production setup as is.