PyPy-STM 2.5.1 released | Hacker News Reader

PyPy-STM 2.5.1 released

morepypy.blogspot.com

55 pointsjocmeh11 comments

> It is a Python in which you can do minor tweaks to your existing, non-multithreaded programs and get them to use multiple cores. You identify medium- or large-sized, likely-independent parts of the code and to ask PyPy-STM to run these parts in parallel. An example would be every iteration of some outermost loop over all items of a dictionary.

This is very neat! Obviously it's not a magic drop-in, but still.

Relevant bit from the docs: http://pypy.readthedocs.org/en/latest/stm.html#transaction-t...

The main point is that at first, your code will be slow (as in bound to a single core) and then you can optimize it to provide better performance. This is as opposed to threads, where your code will be immediately fast, but buggy. The difference comes from the fact that 90% correct is still buggy, but 90% paralellizable is kinda good.

Is there a transaction module that provides a null/no-op implementation for compatibility with CPython?

yes, normal transaction.py should just work :-)

I might be way off here, but does PyPy-STM works at all with the new stdlib asyncio?

I mean, the whole purpose of asyncio is running non-blocking I/O in a single thread and delegate blocking CPU-bound executions to other threads (with the run_in_executor method), but it would be awesome to share the asyncio loop event between multiple threads. I don't know if this is possible though.

asyncio would need some work in order to be trully parallelizable on top of pypy-stm. You would need to go around and eliminate all the global state that all the elements are modifying. But in principle it should work (requires work though)

Amazing! I'd really want to try this out in production if you guys can bump python3 support to 3.3 and have a 3.3+STM version. So looking forward to that!

that should be very easy, since STM is mostly a project that is independent of the interpreter (e.g. you can use it in hippyvm or pyrolog). Good project as a start of contribution ;-)

Is PyPy basically going to replace the regular CPython runtime that people currently consider "Python"?

In PyPy's FAQ, the answer to the question "Is PyPy a drop in replacement for CPython?" is "Almost!"

... followed by a few paragraphs of details about why it is "almost".[1]

Their caveats seem to match up to several of Guido van Rossum's reservations about PyPy back in 2012 (summary here [2])

In the last 3 years there have been significant improvements with PyPy3 and C extension support, so it's getting closer but I would imagine it will take a few more iterations to from "almost" to "definitely".

I personally believe in PyPy's long-term future, they've done amazing work and think it is worthwhile even if CPython remains the reference platform.

[1] http://doc.pypy.org/en/latest/faq.html#is-pypy-a-drop-in-rep...

[2] http://stackoverflow.com/questions/12867263/why-wasnt-pypy-i...

Great work, thanks !

Reply on news.ycombinator.com