Comment by PaulHoule | Hacker News Reader

Comment by PaulHoule | Hacker News Reader

Parent Full thread

I think there is a certain kind of sloppiness in Python culture that holds it back.

For instance, the split between Python 2 and Python 3 has kept from me getting more into Python because I have to choose one or the other in terms of which libraries I want to be compatible with (and the libraries are a big reason I'd want to to use Python.)

If you look at Java or PHP or .NET there is a lot of concern about keeping compatibility between versions, but going from 2 to 3 they changed a lot of APIs that they didn't have to.

The most disturbing thing is that I have been shouted down for years on places like proggit for bringing this issue up but it is only in the last year that I have seen it talked about by pythonistas, and that's a general problem I see with Python, that it ducks the hard questions which Java, in particular, faces before anyone else. (i.e. having a memory model for threads which is mature)

On one hand, the GIL means that Python is not scalable in terms of threads. Today 4 core machines are mainstream, even for phones, and with a language that supports scalable threads, you can get a 4x speedup easily for many real apps.

The other option is go to the Node route, which is balls-to-the-walls async operations in a single process, but (i) the async people also have had a culture of "kicking the can down the road", (ii) it is hard to get programmers who are used to sync APIs to switch to async (this is one reason why the "Modern" interface in Win 8 was DOA... Microsoft was pushing people to use Javascript because rewriting Win32 apps to be async defeats the purpose of using Win32 which is being able to use all the old stuff and not learning anything new.)

You get a 4x speedup with multicore, and get a 100x slowdown for using Python. Furthermore, Python supports multiple cores perfectly well as long as you use multiple processes. Python threads also work fine for I/O. The only thing it doesn't do is shared memory parallelism. Maybe staying within one process gives you an x% speedup compared to the overhead of sending data between processes, but it simply does not make much sense compared to the 100x. If you need that kind of performance you would not use Python.

GIL worries are not founded, most of the time.

Multithreads in Python works just fine for I/O bounded operations. Switch a multithread program to use multiple processes (and avoid GIL issues) is quite simple with the multiprocessing module[1] if you need to achieve 100% CPU usage.

In my experience, complains about the GIL are mostly FUD.

[1]https://docs.python.org/2/library/multiprocessing.html

Some languages manage to have a really nice blend between OS threads and lightweight concurrency. Erlang/Elixir can spin up hundreds of thousands of lightweight "processes" (their word for green threads), which get split up between a number of real OS threads by schedulers. It means you can have great performance on IO with green threads, but also have great performance on CPU bound tasks with honest-to-goodness threads.

Clojure's core.async can pull off the same trick, though I don't know as much about how it works.

There are many ways to get around the GIL. Use PyPy. Use processes. And are you really doing that much computation? Threads on standard CPython are perfectly fine for I/O.

Yes, I really am doing that much computation. Not everyone works on database-dependent web apps.

The GIL sucks. It holds Python back from wider use. I wanted to use Python for the GUI/scripting layer of a high performance desktop application, but dealing with the GIL in a mixed C++/Python multithreaded event-driven architecture was too annoying. When Python code isn't in charge of the main application loop, things get ugly.

These days, I mainly see Python as a source of syntax inspiration for other languages. I say this with the highest amount of respect. I think Python's best qualities - list comprehensions, generators, itertools - are largely independent of its interpreted, dynamic nature. They will eventually find a better home in a language with more industrial-grade semantics.

"The GIL isn't that bad, get over it" is not an attitude that will sustain a language for decades.

PyPy still has a GIL: http://pypy.readthedocs.org/en/latest/faq.html#does-pypy-hav...

Doh. I should have said, "Use Jython."

Numba allows multithreaded gil free numerical code.

Reply on news.ycombinator.com