I think running inference engines is one particular case of that. You can train and tweak your model using python with its wonderful pytorch and numpy. Then at some point when scale and performance matter then this becomes a potential problem.
Companies like Facebook also have huge interests in speeding up python as they have a lot of stacks written in python.
There was a consensus after the Python 3 debacle of “No major breaking changes”. We seem to have lost that because of moneyed interests and that’s sad.
Who does no-GIL benefit? For the majority who use Python for single threaded code, no-GIL will make their code slower because a thread-safe interpreter is always going to be slower than one that can assume ST operation.
For the minority who want parallelism, there’s two other options: OS processes and subinterpreters. If you can use either of these then you will get better performance with a GIL for the same reason.
So no-GIL will only be faster for a minority of a minority who want parallelism but can’t use OS processes or subinterpreters.
Meanwhile everybody else writing libraries has to make sure that their code is no-GIL safe, to support this tiny minority, and if no-GIL ever becomes default then everybody else has to do something to turn it off.
It’s such a stupid idea.
Yes, that was my point.
> For the majority who use Python for single threaded code, no-GIL will make their code slower because a thread-safe interpreter is always going to be slower than one that can assume ST operation.
I'm almost sure the python developers said that they will compensate the slow down with other optimizations, so that you'd never have single-threaded performance degradation version-to-version.
> So no-GIL will only be faster for a minority of a minority who want parallelism but can’t use OS processes or subinterpreters.
One would hope to 1. open new use cases for python thus attracting developers that would have otherwise not given the language consideration and 2. other users could benefit from new optimizations that could be implemented further down the dependency stack.
Of course there's no guarantee that that will materialize, but the idea the adding support to an established, lightweight and well-supported concurrency primitive is so obviously a "stupid idea" shows to me that your (rudely expressed) opinion is entirely self-centered and nearsighted.
I might add that the move from python 2 to 3 was incredibly painful, but I assume most agree (with the benefit of hindsight) that it was entirely correct.
Those optimisations are not there to compensate anything; they will improve performance of single-threaded code with or without GIL.