TurboPython – A Python-to-C++ Compiler | Hacker News Reader

TurboPython – A Python-to-C++ Compiler

tpy-lang.org

26 pointslumpa14 comments

What advantages does this provide beyond what can be achieved with Cython?

People are in love with the idea of a Pythonic language with the speed of an ahead of time compiled language. The issue is without 100% compatibility with existing CPython code, enabling you to use the existing interesting libraries of the Python ecosystem, you’ll get no interest.

There is a trivial subset of python which seems it could be made substantially faster, for example integer / floating point operations. Frustratingly people make attempts and get nowhere, encountering resistance and red tape. (See https://discuss.python.org/t/using-tagged-pointers-to-suppor...).

We even had this project, Pon, posted about 3 months ago (https://github.com/can1357/pon).

> pon is a JIT & AoT native compiler and runtime for Python 3.14, written in Rust.

This is all anyone has ever wanted. It was basically dismissed, and hasn’t been worked on since it was posted. If a project like that can’t inspire interest, I don’t know what will.

> What advantages does this provide beyond what can be achieved with Cython?

Turing completeness aside, it seems to have a different purpouse entirely.

Cython is Python-compatible with its own syntax extensions, and is meant for writing Python modules.

TurboPython is its own incompatible language that has some resemblance to Python and is presumably meant to be used independently of Python.

Author here: you're right, it is a separate language that keeps Python's syntax (it uses Python's ast module for parsing) and as much of its semantics as possible. It does have extensions, but limited to new type annotations and decorators, for things like ownership or dynamic dispatch. It builds to a standalone native binary, but can also be used to compile CPython extension modules.

The source is valid Python though, with compiler warnings where the behavior diverges. In fact that's how it's tested: about 90% of the test cases are run through both TurboPython and CPython, and the output must be byte-identical. However, usually it doesn't work the other way around: not every Python program will pass TurboPython compilation, most will require adjustments to adhere to its stricter rules.

Example: https://github.com/trozen/tpy-lang/blob/master/tests/cases/a... (when run via CPython, the tpy package is a stub implementation of TurboPython's core types)

This example looks interesting:

  def first(items: list[Reading]) -> Reading:    
      return items[0]          # a reference into the caller's list

How the compiler knows and checks if the returning reference is derived from one of arguments? What if a function returning a reference has more than one parameter? I my language I needed to implement a whole notation to allow describing relations between parameters and returned references, has TurboPython something similar?

Author here: there's no notation, the compiler infers that from function body. If there are more parameters, the result is just an union of those, for example:

    def choose(a: list[Reading], b: list[Reading], choose_a: bool) -> Reading:
        if choose_a:
            return a[0]
        return b[0]

the return borrows from either a or b, the caller gets a warning when b is modified while the borrow is held. OTOH returning a reference to an local object is an error; locals can be returned by ownership-passing (Own[Reading] -- NRVO in C++). Currently the analysis happens on AST level, so it has it's drawback. I'm working on a CFG based MIR, to make this less coarse, though that's still a while since it's ready.

> there's no notation, the compiler infers that from function body

Good. My language has such ability too (if return type is specified as auto). But I do need a notation for some cases, like if no body definition is available or for recursive functions. How do you deal with recursion?

> the caller gets a warning when b is modified while the borrow is held

Why a warning and not an error?

Recursive functions are handled partially ATM, for common cases like find(l, i) where the function is called recursively with the same argument. The CFG based approach should solve that by walking a call graph until fixpoint (with some budget). For body-less functions (ones that are defined in C++, e.g. builtin functions like min/max) the language has annotations to provide such information:

    @cpp_template("::tpy::min_key({0}, {1}, {2})", borrows=("a", "b"))
    def min[T, K: Comparable](a: T, b: T, key: Fn[[T], K]) -> T: ...

(the result may borrow either a or b)

Current approach is to warn because the checks are less precise than what I'd like them to. The idea is to keep it that way for now, migrate to CFG in the future and enable it in a strict mode.

It reminds me of PyPy (https://pypy.org/)

It works as a pure Python runtime, but when comes with issues when integrating with existing libraries who have C/C++ extensions, which is a big one for modern use of Python (Numpy, pandas, you name it)

Nice approach for memory safety! I have my own memory-safe programming language (see my submissions), but it has syntax similar to C++. Doing so with Python-like syntax and other Python-like elements seems for me to be a little bit unusual.

It talks alot about how it's different from python and not at all how it's different from rust (the language that it is actually similar to). Syntax is ultimately very superficial and doesn't really matter all that much.

Coming from C++/Rust

https://tpy-lang.org/docs/guide/cpp-rust/

>no garbage collector, no automatic reference counting, no GIL

The page is triggering my AI alarm.

If you want an AoT compiled almost-Python, why not use Mojo?

Mojo has just python-like syntax, but its goal was never to be a subset or superset of python nor to be compatible with python. Which is not the case of TurboPython or Cython.

Reply on news.ycombinator.com