Great talk on the Python GIL [video][slides in comments]

I posted this earlier: http://news.ycombinator.com/item?id=654560

Some really good sleuthing, but unfortunately no conclusions on what to do about the problem.

Lots of folks have started to tackle this in the past and given up in horror.

From Python 2.6 there is a nice multiprocessing library which based on OS processes. It's more heavyweight from the OS point of view but you don't need to worry about issues with GIL. I'm using it on an EC2 image with 8 cores and it rocks.

Plus it has the added benefit that if you design your architecture with queues as your distribution primitive you can scale across multiple machines pretty easily. There's a mostly drop-in interface [1] to AMQP that mimics the native Python Queue interface.

[1] http://www.lshift.net/blog/2009/06/11/python-queue-interface...

very interesting and thorough explanation, + easy to understand!

recommended for everyone who wants to know more about python, threads, ticks and signal handling.