Why? Java has no GIL and runs faster than Python.
Lots of Python scripts only run for a short time. Code in the REPL needs to return results really quickly. The .NET JIT added significant startup costs to the point where the IronPython team had to add an interpreted mode for first runs before handing off the JIT. This is complex and arguably takes time away from CPython compatibility work.
It seems that the .NET team is coming around to the concerns around startup time, but I don't know if they have landed on an AOT solution. It probably will help executables running on .NET Core more than scripts running under IronPython. I've not kept up with JITs very much, but I don't think this pattern is limited to the .NET JIT, and is a general trade-off. I feel like JavaScript JITs may be closer to what IronPython would have wanted though.
It might be possible to retain current interleaving semantics, through research-level techniques such as transactional memory, but these do not seem likely anytime soon despite a lot of work.
It's also unclear how tractable it is to retain current garbage collection semantics without a GIL, so that could have to change as well with similar issues and similar possible solutions which seem unlikely to land anytime soon.
How is this currently avoided? My understanding is that anything that accepts a callback / lambda is potentially subject to interleavings as the code can call a C extension which releases the GIL. Moving code to C extensions is often recommended when Python performance is brought up. Am I misunderstanding something?
> It's also unclear how tractable it is to retain current garbage collection semantics without a GIL
Why is it important to retain current garbage collection semantics? Why does Python prescribe a certain garbage collection implementation and does not allow for others like Java?
Consider the expression "a.b.c". Where are the function calls? In other words, which of the ".{name}" return a value from some namespace and which call code? (I'm not asking about which functions are called. I'm just asking about where functions are called.)
Java semantics, including the type system and the required declarations, tell you where the function calls are. One consequence is that they're always in the same place.
That's simply not possible in Python because the answer isn't known until the expression is evaluated AND the answer can change every time the expression is evaluated.
Though it is worth pointing out that Java has a well-defined memory model, and Python doesn't. So the above is true for CPython, but it might not work in other implementations, or, for example, you might have a library providing data structures, and it might not hold for those, either.
How is this achieved?
> Though it is worth pointing out that Java has a well-defined memory model, and Python doesn't.
Wouldn't you have to at least issue an mfence when a thread enters / exists the GIL?
How do you prevent interleaving of different threads then?
> and must remain that way to maintain compatibility.
Why? Python 3 broke a whole lot of applications and libraries.
The GIL.
> Why? Python 3 broke a whole lot of applications and libraries.
Sorry, I mean it must remain that way if it is to maintain compatibility. We’re assuming that breaking compatibility is unacceptable.
How does the the GIL prevent interleaving? My understanding is that if an "operation" takes several steps, eg. reading from a dict then IO or a C extension and finally writing to a dict the GIL will not make this atomic.