PyPy: Transactional Memory (II)
morepypy.blogspot.com
morepypy.blogspot.com
Would you post more info about previous attempts at this? I was about to create precisely that. Literally today.
But if it's fundamentally unworkable...
Having a serialization layer in distributed applications is a pretty vanilla requirement... I am also confused about the "unworkable" comment.
The "multiprocessing" module (which I have heard some not-so-good things about) automatically serializes Python objects over byte streams using pickle or some such thing.
Would someone please help me out and point me at a way of efficiently exposing a read-only block of memory to Python?
EDIT: Looks like "MemoryView" introduced in 2.7 is the answer.
as for shared memory, have you tried just mmap'ing the node from /dev/shm ?
EDIT: I just saw your edit. That's what I'm doing --- shm_open(), ftruncate(), then mmap(). A little annoying that it doesn't automatically kill the shared memory when the program terminates, but still...
The more I think of it, the more I think that "high concurrency with local state on one machine" is a use case that is not going to get more popular in the future.
I believe a big part of the uptick in interest in concurrency is an uptick in the use of distributed systems. With distributed systems, all your authoritative state is stored outside the process anyway (at least if you want it to be fault tolerant, which you do if you need it to be of any significant size).
And of course once you have more than one machine you must deal with serialization. Also, I'm not sure what the kind of work there is where "serialization is prohibitively expensive" but Python is still a natural choice.
regarding "Python is still a natural choice" - don't confuse python as a language and python (CPython) as interpreter. You would not use CPython for any performance-critical tasks probably, at least not without spending significant time rewriting pieces to C, but PyPy is quite usable in some scenarios and the list is only to grow.
Such as...? A lot of the big users of cray-type systems were for scientific uses. AFAIK a lot of them are seriously looking at cloud or commodity-type clusters as the problems get bigger.
Anyway I am curious if PyPy is aiming at some specific problem that I don't know about. For "web stuff", I think what is proposed is perhaps overly complicated.
If you want a large read-only set of data to fit on one machine and need high performance, Java, Python and the like aren't great choices because they don't give you much control over the memory layout. Python is probably better because you could write a C extension. But PyPy itself is not optimized for memory size (in fact I think it uses more memory to get speed).
Relevantly, CCP say the idea of EVE is "incontheivable" without Stackless Python[0], which doesn't have STM, but does go in a similar direction (co-operative multitasking) to solve the same problem.
Of course this problem becomes moot if the database supports auto-sharding over multiple machines, but SQL databases generally have bad support for sharding.
I completely disagree. Unless you mean something very special by "complex".
IPC, even complex IPC, is definitely workable. For example, most web browsers today have a split process model. That's as real-world an example as you can have, and it's definitely complex. It works great though.
Furthermore, all that this is, is the model of no shared state/message passing (that the talking entities are in separate processes and not threads is not fundamental here). That's a very clean model of parallelism, much better than shared state. Lots of languages use the no shared state/message passing model, it's definitely not "essentially unworkable."
Are you really saying that the actor model and in general message passing/no shared state is "essentially unworkable"?
Sometimes you deal with something that is more time-critical and serialization of objects (like a complex graphs of objects) would kill a lot of performance benefits. I'm sure it takes a lot of effort to make a browser (not to mention a multiprocessing one), but a lot of those complexity are not because the task at hand it's hard -- quite the opposite it's just lots and lots of easy tasks crammed together in a big pile, usually C++ pile with tons of history on top of it.
Note that serialization is not necessary in this model, for example you can have actors in the same process, and transfer ownership of objects. So there is no shared state, and they are logically passed as messages, but there is no copying cost to doing so or need to implement serialization.
That is, one big problem with TM is I/O, like doing network read and write. But if you are already writing event-driven programs, you return to event loop whenever you do network read and write, and at that point TM implementation can do smart things. If TM is sold as "write threads without locks" this can't work, but if TM is "write events but with more parallelism" this seems possible.
Actually I am quite surprised that this model was not heavily investigated already.
That said, a lot of people seem to dislike threads. The problem isn't with threads, but rather having to shoehorn every attempt to parallelize your program into a threaded model. I would feel similar ire if I had to shoehorn every attempt to parallelize my program into an event model.
Threads are a way you can reason about and simplify your program. I've written programs where threads make the program easier to reason about rather than harder. Because of this, telling me your method is better than threads makes no sense to me - for what class of problems?
I want to hear about this.
We went with: queue per ISP for emails to send out, then each server that could send email to that ISP had its own thread. Each thread would take X emails out of the queue every 10 seconds, send them, then sleep for any leftover time - or not if there isn't any.
Most of the time is spent sleeping or on I/O - the point of threads wasn't for CPU usage, it was because it made everything so simple to implement. The only synchronization issues were taking emails out of the queue and putting them back somewhere if they failed to send.
All of the logic for each thread in its main loop read like a normal, sequential program. I guess you could call this an embarrassingly easy problem, but part of why it's so easy is it maps so easily to threads.
At work we recently wrote something for handling automated translations of content via our external review providers. Occasionally some stuff-to-translate gets pulled from a database, we figure out what it needs to be translated to, and job orders get dropped into queues for each (provider, from-lang, to-lang) tuple. Once the queue is processed, the translated content gets pushed back into the database.
One fairly major catch: each of our translation providers has different numbers of concurrent translation requests - we can do two simultaneous English->Spanish translations, say, but four English->French ones. So I wrote it such that each (provider, from-lang, to-lang) queue is serviced by its own threadpool that has with an upper bound on its active threads equal to the maximum that the external translation provider can concurrently handle, so we get in-flight management for free. (This could have ugly consequences in overloaded cases with tons of providers/tons of in-flight requests, but the machine it runs on is dedicated to this process and, as we tend to do, the JVM it runs on is provisioned with approximately eight hojillion bytes of RAM.)
There are certainly other ways to accomplish this task, but threading and threadpools made it conceptually a lot simpler to reason through. (Though evented was the first thing I thought of, we couldn't really do something using it--both because Java's support for it is poor and because of the nature of some of our translation providers. For one at least, we have to repeatedly hit an HTML page and scrape it to find out when our translation job is done!) We could functionally treat each module as a discrete case - the simple queued interface let the people writing the translation handlers treat it as if it was a single-threaded application. That's actually exactly how we wrote it, too: I told the other person on the project to just drop the class they were writing into main() and make sure it worked, while I built the infrastructure in which it'd actually run as a daemon. He didn't have to care about the threading, while it gave us resource control and ease of expansion.
In Python, threads are "enough" to implement event-based concurrency, as well as asynchronous I/O and calling out to long-running C functions for number crunching.
Beyond that, the main reason to remove the GIL and allow true multicore parallelism is purely for efficiency reasons. The catch is, if I'm serious about efficiency, I probably won't be using Python. Or at least porting only my CPU-intensive functions to C.
Of course, if PyPy eventually provides Java-like speeds, maybe it makes sense to go this route, as Python could start to be depended on for efficiency, so the gains of true parallelism would be visible. But removing the GIL from CPython seems kind of pointless, since the interpreter has so much overhead regardless.
As far as I understand, this is exactly the reason why the PyPy team went for the STM route: they want to guarantee the same semantics of CPython with the GIL, but with more parallelism.
The idea here is that most of the time threads do not conflict, which means that they do not mutate objects while other threads are accessing them. Based on this assumption, you can optimistically run a snippet of code assuming that no conflict will happen, but if there is such a conflict all the effects of the code (the "transaction") are rolled back and the code is run again.
Since the interpreter can decide where to put the transaction boundaries, it can do it where CPython would acquire/release the GIL. This would give basically GIL-equivalent semantics.