Hacker News Reader

Comment by ledauphin | Hacker News Reader

in my opinion, getting rid of the GIL is the one thing that Python can do to stave off Julia's eventual dominance of numeric computing. Not indefinitely, but by at least 5 years.

There are still many cases when data scientists don't have a C extension and need to run pure Python functions in parallel, and the GIL makes this an order of magnitude harder (and sometimes slower).

I think the core Python devs have gotten rather too conservative in the language's old age.

View on HN

In terms of numerical computing I don't think this is going to be a battle of core language speed, Python has already lost here. The question is going to be what are the advantages of a language designed with a focus on number crunching vs a multi-purpose language with some number crunching extensions.

I mostly do scientific computing but I really enjoy the toolchain of an extremely popular multi-purpose programming language (Python) + C extensions where necessary for speed. The reason is that many tasks these days are not just pure number crunching. If I need to start up a quick web server, do some web site scraping, basically anything can be done easily from Python. It's unlikely Julia will ever be able to catch-up in these areas.

For numerical computation compiled code is 10x-1000x faster; even without the GIL Python cannot compete without dropping to C/Fortran (which typically releases the GIL anyway)

wouldn't removing the GIL cause all sorts of sudden race conditions and regressions in existing python code? I think there is a valid argument to keeping the GIL just for simplicity's sake

From my understanding, race conditions already exists today as the GIL is made to protect the internal Python structures. It only made it appears more infrequently.

> It only made it appears more infrequently.

It makes some operations atomic which is quite relevant.

Fewer than people expect. For example, the += operator looks atomic, but under the hood, it's made up of multiple python bytecode instructions, so the GIL won't apply here.

> For example, the += operator looks atomic, but under the hood, it's made up of multiple python bytecode instructions, so the GIL won't apply here.

Sure, but any call to a native function which doesn’t release the gil (which is most of them at least for the builtins) is currently atomic so something like dict.setdefault.

And note that this is mentioned in the faq.

Those would have to remain atomic by taking explicit more granular locks.

Obviously?

But the point is currently they do work, they are thread-safe and they are correct in the strictest sense of the word, per-FAQ: https://docs.python.org/3/faq/library.html#what-kinds-of-glo...

So as the original commenter notes:

> wouldn't removing the GIL cause all sorts of sudden race conditions and regressions in existing python code?

I have never used Julia before. What about it makes seems so inevitable?

It isn't inevitable, but some people promoting Julia like to pretend that there is some sort of competition for mindshare happening. There are three basic markets for numerical computing: hobbyist, academic, and commercial. The hobbyists will always remain in Python because it is good enough and there is little benefit to learning a new language for casual numerical computing (this also applies to undergrad-level academics, which is closer to hobbyists than deep academics.) In academics it is possible that Julia will replace R for a lot of use cases, but it is unlikely to make much progress in displacing Python outside of math-heavy fields: a biologist or chemist will stay with Python because of the ecosystem and its applications outside of pure numerical processing code. In the commercial world the race is already over and Python won, it will continue to grow in this role due to simple inertia and because for cases where numerical computation speed actually matters a company can hire people to write the code in something even faster and more efficient than Julia.

> Is possible that Julia will replace R for a lot of use cases

R is a interactive statistical programming language that acts as a frontend for more performance languages. AFAIK interactivity is not the strongest points of Julia at the moment.

>interactivity is not the strongest points of Julia at the moment

Not sure what you mean. Julia has the exact same notebook environment (Jupyter) as R and Python. Fun fact: the “Ju” in Jupyter stands for “Julia” (the “pyt” and “r” stand for what you think).

Few people in the R world use Jupyter environment, they are generally inferior to use the Rstudio environment. This might change in the future though, VSCode and DataSpell are becoming really good. Jupyter Labs are slowly improving too.

Julia has some excellent interactive front ends.

For example, take a look at the reactive Julia notebook, Pluto.jl: https://plutojl.org/plutocon2021

I am not sure I agree with your hobbyist point. I do programming as a hobby, and I'd prefer doing it in Julia over Python anytime. I can understand that commercially you wouldn't risk adopting a new programming language like Julia when Python has a bigger mainstream ecosystem with solutions for most problems people have ran into, but for hobby I do not have these constraints; I have much more fun programming in a cleaner and more modern language like Julia withstanding its other benefits.

I think Julia may be the Rust of the Number Crunching world these days and you'll get similar advocacy while the old crowd keeps trucking along with c++/python

Reply on news.ycombinator.com