Would it be feasible to keep a set of Python interpreters around at all times and use a round robin approach to feed each already-on interpreter commands then perform an interpreter environment cleanup out-of-band after a task is complete?
There's also nailgun for Java which sounds like it works a little differently: http://martiansoftware.com/nailgun/
Also, could it be that Intel's cache hierarchy plays some really smart tricks behind the scenes, to make this fast?
The idea of keeping persistent interpreters doesn't really work for Python because the interpreter is full of state in places you'd never expect -- it's hard to reset the interpreter to a sane state after it ran some unknown program.
https://github.com/tbug/aiochannel
I also think David Baezlys curio has a wonderful way of explaining the same concept
http://curio.readthedocs.io/en/latest/reference.html#module-...
Asyncio has enabled his sort of programming a lot easier. You could do the same thing I imagine with multiprocessing and Threadpools I imagine