PyPy: the other new compiler project | Hacker News Reader

PyPy: the other new compiler project

lwn.net

52 pointscorbet11 comments

Off-topic: I was intrigued by the 'ATS' in the shootout (don't remember it being there a couple of years ago).

I had followed the link posted by rodion_89 (2008 blog), into http://shootout.alioth.debian.org/#acceptable (nice opening and closing) and http://shootout.alioth.debian.org/u64/performance.php?test=b... - where 'ATS' sits at #3 (behind C and C++, edging out Java6).

The code (e.g. http://shootout.alioth.debian.org/u64/program.php?test=binar... ) does not look too ugly, but apparently it may be twice as verbose as C++ ( http://shootout.alioth.debian.org/u32/ats.php ).

So, on to http://www.ats-lang.org/ :

ATS is a programming language with a highly expressive type system rooted in the framework Applied Type System. In particular, both dependent types and linear types are available in ATS. The current implementation of ATS (ATS/Anairiats) is written in ATS itself. It can be as efficient as C/C++ (see The Computer Language Benchmarks Game for concrete evidence) and supports a variety of programming paradigms that include:

- Functional programming. [...]

- Imperative programming. [...]

- Concurrent programming. [...]

- Modular programming. [...]

PyPy while great news for Python, is still a snail compared to another high-level language JIT, LuaJIT:

http://shootout.alioth.debian.org/u32/benchmark.php?test=all...

edit: bit of background: Lua is a minimalist dynamic language. It can be used anywhere in your web development request stack since mod_lua was included in Apache 2.3.

And it also has nothing to do with it from an architectural point of view. LuaJIT's code is not generally reusable. It's highly optimized, often platform specific C and assembler code. PyPy is an experimental (in the good way) and successful attempt at using a subset of python to make a faster python interpreter than the native C interpreter, while in the process creating a platform to create other optimized dynamic languages interpreters. It's code is highly general and readable, especially compared to something as specific and low level than LuaJIT. In the process, the devs have created a new language, rpython, that is as fast as C while being much more expressive.

Lua is also a much simpler language than python, and therefore a much better target to fine tuned optimizations.

You could almost say that LuaJIT is a pragmatic project, and pypy an idealistic one. And they're both succeeding. But they don't have the same goals at all.Please dont compare them without giving a bit of background. It's very misleading.

Do you have any references for your "as fast as C" part? That seems like a very dubious claim.

If your ultimate goal is performance, I believe PyPy is fundamentally flawed because it already introduces two levels of indirection that cannot easily be removed. Of course their goal seems to be to simply be faster than the python interpreter and remain maintainable at the same time. They appear to be succeeding at that and I congratulate them on that.

LuaJIT takes the right approach w.r.t. performance. The interpreter for LuaJIT 2 is well-optimised and not actually that hard to understand. This goes together with a well-chosen set of interpreter instruction set and intermediate representation. All these layers require careful design, and Mike Pall didn't come up with this over night.

Here's how RPython is "faster than C":

1. PyPy's Python interpreter is written in RPython, a subset of Python.

2. CPython is an interpreter written in C.

3. When compiled into a JIT, PyPy's Python interpreter is now generally faster than CPython, sometimes by a wide margin.

4. Code written in RPython is faster than code written for a nearly identical purpose in C. The C code in question has been the target of aggressive optimizations for years while the RPython code is relatively immature.

Now, for many applications I'm sure C remains faster than RPython. But for the application RPython is designed for--writing an interpreter for a dynamic language--I think it is safe to say that RPython appears faster than C.

The fact that the PyPy's interpreter is cleaner, safer, and more maintainable than equivalent C code is also a huge win. Again, the point of the original post is that RPython could become an simple implementation language to create fast interpreters for dynamic languages. It could also become a target language for other compilers.

Anyways, as others have pointed out, Lua is a considerably simpler language than Python. LuaJIT is a really remarkable piece of engineering, especially since it was written by one guy, but it's really a completely different project than PyPy. Its aim is to make Lua--and only Lua--extremely fast. If PyPy's Python implementation gets no faster (which I think is unlikely) the potential ability to easily create new dynamic languages that are similarly speedy is itself an amazing achievement.

I find it interesting that people say Lua is a "simple" language, as if that's a flaw. Also, simple in which respect? To optimise? To parse?

Note that this is no fanboyism. It's just that these performance claims are mostly phony anyways. How much % of general python code is in RPython?

It's good to see this kind of progress, for both python and lua, as it opens up room for dynamic languages in new areas and helps to change the mindset of some people. However, this picking on each other is more than expendable IMHO..

I don't think anybody is picking on each others. The two projects are way too different to be compared IMHO.

Also being simple isn't a flaw in my mind. Lua is quite a beautiful language. And theoretically much more closer to my likings than python. However, for practical programming, i prefer python, just because it's a 'good enough', very comfortable to work with language that has a ton of useful and well done libraries. This isn't the case of lua, probably because it's purpose isn't to be used as a general language, but to be a real scripting language to be embedded in bigger applications.

> 4. Code written in RPython is faster than code written for a nearly identical purpose in C.

This is where your argument falls down. Selection sort and heap sort are both written for the identical purpose, but due to their different designs and different complexities, a heap sort written in ordinary Python would outperform a selection sort written in C for a great many inputs. This does not prove that Python is faster than C, it simply proves that two programs, even written for the identical purpose, can achieve that purpose in vastly different ways, and those differences can greatly impact the performance of those programs.

A downmod but no counterargument? Sad.

apparently it can be faster than c.

http://morepypy.blogspot.com/2008/01/rpython-can-be-faster-t...

While true, lua is more directed towards another audience, namely a small, embeddable scripting language with as little overhead as possible (IMO). This is not meant degrading, but i guess it has some implications. Lua is great for embedded scripting though.

Reply on news.ycombinator.com