Hacker News Reader

Comment by lzauz | Hacker News Reader

>Gil free python would break a lot of oss packages written with the C API that are only thread safe because if the GIL.

So make it a runtime option.

It's getting tiresome that python performance is suffering because some can't be bothered to write thread-safe software. In 2022.

View on HN

This comment demonstrates a tremendous amount of naivete about the Cpython runtime. After PyObject itself, the GIL mutex is probably the next most important data structure in the entire codebase. It's not "someone not being bothered to write thread-safe software." It's not something you can hide behind a flag. It's central to the entire cpython data model and any library which relies on releasing the GIL.

The closest anyone has come to removing the GIL is the Gilectomy project by Larry Hastings, and it's unlikely to ever be upstreamed unless it could be somehow made to work with libraries that rely on assumptions about GIL mechanics (eg numpy).

> The closest anyone has come to removing the GIL is the Gilectomy project by Larry Hastings

It was Sam Gross and he more or less achieved it:

https://mail.python.org/archives/list/python-dev@python.org/...

In C which Python extensions use, it hasen't gotten any easier to write thread-safe software "in 2022"

But it has become more necessary and useful, so even C programmers better get used to it (and some tooling to help catch most mistakes).

>so even C programmers better get used to it

Or else? It's not like they're not trying their best - or don't spend the level of effort that they and their companies are willing to take...

I guess it would have made more sense for me to say "Python C extension developers".

I don't find multithreading in languages without particular support easy at all, but I have become better at it. It is possible and sometimes necessary. It seems like the prevailing attitude in the Python ecosystem is weird, a kind of sour grapes thing, i.e. "Python doesn't have good multithreading support, but multithreading is ugly and error-prone anyway and the alternatives are almost as good or better".

Usually when I've needed more parallelization I've allowed more processes and for slow methods, there is threading available (this doesn't overcome the GIL but allows those methods to independently operate). It seems like the biggest reasons to focus on removing the GIL are single-process applications or machines where memory is constrained (so you don't want tons of processes consuming it all). Are you in one of those situations or is there another scenario that is impacted by the GIL?

Reply on news.ycombinator.com