Hacker News Reader

Comment by chrisseaton | Hacker News Reader

I wonder if you could completely elide the CAS needed by userspace locks (the bit which thrashes the cache I presume) while there is only one thread running. Then when a new thread is created for the first time (if it ever is), pause the program and promote and acquire all the locks.

Like how the JVM assumes classes are final and removes virtual calls until it first sees a subclass.

View on HN

Late in the talk he discusses strategies to avoid thrashing that amount to making locks as local as possible, ideally such that they behave the same way they would in the current unsynchronized environment. The big problem is that all the locks are global, so a change to a lock triggers a gigantic cascade of cache invalidation (and voided branch prediction, etc).

It's a great talk; the video is really worth watching. At the end, he says he welcomes anyone to join him in the sprints, but basically don't bother unless you really know cpython and multithreading programming because this is very advanced stuff.

Right, but I'm suggesting why bother using atomic operations and cache invalidation if there is known to be just one thread? Which is the case we want the lowest overhead on. Why not only start doing that stuff when the second thread is started.

Ok, I think I see the point you're making, but what I took from Hastings talk is that devolving locking to as local a scope as possible is doing just that: wherever possible, locking in an analytically single threaded context which is trivial and basically cost-free.

The cache invalidation isn't done by cpython, it's done at the the chip level when global atomic incr and decr instructions are triggered. Hastings was pretty clear that the performance hit was very closely tied to the way that modern processors depend very heavily on caching, pipelining, branch prediction, etc.

I think that the kind of implementation you're suggesting, where the VM adopts a locking strategy based on whether or not it's actually multi-threaded, might be possible, but sounds like it conflicts with Guido's edict that no GIL-less solution will be accepted that overly complicates that cpython codebase.

I'm probably making a hash of the talk, but I saw it in person, and it was surprisingly clear to someone like me who isn't a serious C programmer, but remembers their class in operating systems and kernel design.

The complexity requirement is the tricky one then, yes. I think I'd disagree with Guido there - a little complexity in the VM managed by a few experts is probably a good cost to accept for millions of users.

It's not Guido's point. He means that CPython, being the reference implementation, must remain easy to understand.

Although, this suggestion is kind of close to Hasting's implementation plan of having separate compilations for GIL and GILless python, so that the default Python with a GIL continues to be the norm in most cases, but a GILless python with extensions taking advantage of true threaded parallelism is available as a 'high-performance' python for scientists and number crunchers and such.

Some JVM do exactly that. The optimisation is called biased locking and would likely work in python as well.

I think that was one of the suggestions from the after-talk conversation. The trouble is that you have to do it on thread creation, which is sort of a cheesy way to ensure single-threaded performance doesn't go down: it still leaves multi-threaded applications performing worse than under the GIL, which isn't super helpful. Ideally you'd do it on the first access to the object by a second thread, but there's no particular interface that makes it clear that's what's happening: the second thread just knows about the object somehow, and reads it at some point. And there are enough global variables in Python (e.g., every module) that simply promoting it when it's visible to multiple threads is too aggressive.

There's a lot you can do. There's not much you can do in cpython as it currently stands without breaking a lot of code, especially when it comes to extensions.

I work on a new Ruby interpreter, and our solution to this problem has been to interpret the C code of extensions. That way we can give it one API, but really implement another.

Sorry if I keep responding, but there's a lot of interesting points in your question.

> Like how the JVM assumes classes are final and removes virtual calls until it first sees a subclass.

AFAIK, the python VM doesn't do any JITting at all. A runtime decision in the VM about handling locks is impossible at this point, I think.

Well the user space locks he discusses already read a variable and branch between uncontended and contended cases. Specialising speculatively for single threaded would just be a third cannot-be-contended case. But yes it'd work better with a JIT - yet another reason to write one.

There is a version of Python that JITs -- I forget it's name at the moment, though.

PyPy

there are several. we (microsoft) are also trying to get an api into cpython so that anyone can use to build JITs for python. our sample implementation is at:

https://github.com/Microsoft/Pyjion

pycon 2016 talk:

https://www.youtube.com/watch?v=1DAIzO3QXcA

Sort of.

    if (threads)
         lock(...);

should still be pretty quite a bit faster than fine grained locks everywhere, since your atomics aren't broadcasting and shooting down cache lines everywhere.

Reply on news.ycombinator.com