Hacker News Reader

Comment by santiagobasulto | Hacker News Reader

There's a great talk by David Beazley (I don't remember which one) that he basically explains that removing the GIL isn't difficult at all. After all, it's just one lock.

The issue is all the libraries and packages that were built around it. It's now been YEARS of building on top of the GIL.

So anyways, PEP 703 is a a nice effort, but I doubt we (everyday mortals) can enjoy it. Meta, and big companies with a specialized team might be able to exploit it by making sure their entire stack is GIL-free.

EDIT: My bad, the talk is from Larry Hastings: https://www.youtube.com/watch?v=P3AyI_u66Bw

View on HN

> Meta, and big companies with a specialized team might be able to exploit it by making sure their entire stack is GIL-free.

Not really. Most of the popular libraries will have nogil versions ready for sure (numpy, pandas, pytorch, etc). Server applications will probably benefit from being able to serve multiple requests in a real multithreaded environment without many changes (in fact, I bet a lot of HTTP libraries will see simplifications due to not needing to deal with processes for real parallelism). Even packages without changes will probably have at least a small degree of thread safety (for example, two instances of the same object should be independent and could be used in different threads) that allows users to still leverage parallelism.

> So anyways, PEP 703 is a a nice effort, but I doubt we (everyday mortals) can enjoy it.

I'd suggest we can be more optimistic. There is a lot of python, regular users write daily, where we delegate orchestration to established libraries - asyncio/web-frameworks/pytorch. The GIL limits how much they can parallelize your code, and its removal will help with that.

Sam Gross has worked vs numpy and other modules to make it work well in practice

“Just one lock” is kind of misleading if decades of software has accumulated that is predicated on its existence

2016. A lot has changed since then and AFAIK numpy and other most popular libs with bindings in C have an experimental branch that is already nogil compatible.

Not saying he isn't wrong, though.

Reply on news.ycombinator.com