Hacker News Reader

Comment by xscott | Hacker News Reader

I don't understand why people care about the CPython GIL. For computationally intense stuff (numerics) running in the interpreter, the language is generally 60 times slower than PyPy and often 100 times slower than an equivalent C program. That means if you want performance, you would need to dodge Amdahl's law and have 60-100 processors for a hypothetical GIL-less CPython to match a single threaded program in PyPy or C.

View on HN

If all it was used for was scripting it would be fine, but what is strange is that Python has emerged as a major language for complex data analysis. So I am sometimes confronted with partial solutions in Python that solve a piece of the puzzle that is purely numerical (so all runs on numpy etc) but then extending it to actually solve the real problem ends up either extremely awkward (the whole architecture getting bent around Python's limitations) or being a complete rewrite. It's frustrating that the ecosystem for certain problems is so bent towards a language that is unsuitable for large parts of the job. (mainly, I would add, because it sucks the oxygen away from alternatives that would be better in the longer term).

> it sucks the oxygen away from alternatives that would be better in the longer term

It's always frustrated me that the computing world isn't as it should be. Python as a language seems to have found a sweet spot for accessibility to a lot of people, but CPython the interpreter steals all the air from PyPy, which is a vastly better implementation.

PyPy has a GIL too though.

The PyPy FAQ addresses this. To my eyes, it looks like this is partly to provide the same promises CPython does for C extensions, and partly because their transactional memory approach hasn't gotten the attention it would need to succeed. I think both of those show CPython stealing the air from PyPy.

https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil...

You cant both claim that it is major language for complex data analysis and that it is "unsuitable for large parts of the job". If it is a major language for complex data analysis it is apparently not unsuitable. It can be not optimal/not the best/bad but it cannot be unsuitable, because then your first statement isn't true. Which one is it?

It's a great language for exploratory prototyping of complex data analysis and a terrible language for robust implementation of complex data analysis. Well-managed organisations prototype in Python but take the time to rewrite the prototype for production use; poorly-managed ones skip step 2.

What language do you rewrite it in so?

It's typically pretty easy to turn Python+numpy into performant Julia.

I remember reading this comment:

>I recently tried to migrate my R code to Julia. Even though I already knew R data.table is faster than DataFrames.jl, I was totally blown away by how slow Julia is. So I quickly gave up. I think I will have to write unavoidable hard loop in cpp, which I really don't want to do...

https://news.ycombinator.com/item?id=27404910

Apparently it is working for some, https://juliacomputing.com/case-studies/

In the case I was thinking of, mainly Scala (we were a very JVM-oriented place and used Spark clusters for our big data stuff).

Does this seem really inefficient to anyone else? Why is there not a language that can do the prototype and offer a high performance ceiling? Julia seems like a candidate.

There's a language design tradeoff - Python is is the result of the designers almost never saying "We should do this to make Python faster, although it would make the language slightly harder to program in."

This even goes as far as choosing not to make improvements that improve performance but make the implementation more complex (for example adding a JIT), which allows Python to continue to add language features.

It doesn’t work like that in any other industry, why should it work for software? Mechanical engineers don’t put prototypes into production, why should software engineers?

we are writing software, which is a totally different category to mechanical engineering.

The language isn't the hard part. One of the more programming-oriented data scientists was actually happy to write Scala to start with, but honestly getting from a prototype to a production-quality implementation is pretty much a rewrite anyway (e.g. you have to consider all the error cases).

They're saying that Python has gained a reputation for being the language for complex data analysis, but in fact it is objectively "unsuitable to large parts of the job".

If you're doing complex data analysis in python, the actual number crunching isn't happening in python it's happening in a C extension. Using numpy or tensor flow, you're really just calling C code.

And Fortran code (many parts of numpy). C doesn't have a complete monopoly on high-performance code.

What is the advantage of Fortran over only C in numpy’s case specifically?

Fortran has the more optimized libraries (blas) and is also capable of optimizations that C isn't (guarantees that pointers dont alias).

Modern C has the restrict keyword for that. There isn't any competitive advantage left for Fortran over C or C++, the only reason why Fortran is still part of the modern numeric stack is that BLAS, LAPACK, QUADPACK and friends run the freaking world and nobody is ever going to rewrite them to C without a compelling reason to do so.

I'm aware of the restrict keyword, though its usage is not typical. It enables an optimization that fortran has by default, that was my point.

And yes, I already said that packages like blas have had massive amounts of work put into them and have lasting power.

I didn't downvote you, if that's what you were hinting at, however your phrasing was more likely to be interpreted as "C can't do that" as opposed to "C doesn't default to that".

It seems we agree, after all.

I believe we do.

Fortran's primary competitive advantage over C and C++ is that it's a conceptually simpler language that's easier for humans to use safely and effectively for numerical computing tasks.

It's not just being tied to BLAS and friends. Fortran lives on in academia, for example, because academics don't necessarily want to put a lot of effort into chasing wild pointers or grokking the standard template library.

For my part, I'm watching LFortran with interest because, when I run up against limitations on what I can do with numpy, I'd much rather turn to Fortran than C, C++, or Rust if at all possible, because Fortran would let me get the job done in less time, and the code would likely be quite a bit more readable.

Note that while BLAS and friends aren't getting rewritten in C, there is an effort underway to write replacements in Julia. The basic reason is that metaprogramming and better optimization frameworks are making it possible to write these at a higher level where you basically specify a cost model, and generate an optimal method based on that. The big advantage is that this works for more than just standard matrix multiplies. The same framework can give you complex matrices, Integer and boolean matrices, matrices with different algebras (eg max-plus).

That's a cool idea. I don't know how realistic it is to achieve performance parity, but the "generic" functionality is definitely intriguing.

The initial results are that libraries like LoopVectorization can already generate optimal micro-kernels, and is competitive with MKL (for square matrix-matrix multiplication) up to around size 512. With help on macro-kernel side from Octavian, Julia is able to outperform MKL for sizes up to to 1000 or so (and is about 20% slower for bigger sizes). https://github.com/JuliaLinearAlgebra/Octavian.jl.

Except "modern" C does not prevent one to misuse restrict and doing so is UB.

Also, when Travis Oliphant was writing numpy he liked the Fortran implementations of a lot of mathematical functions, so it was an "easy" borrow from there instead of reimplementing.

I think the OP's point was that it forces you to do all the number crunching via those extensions, and if what you want to do doesn't match up with their APIs it's a bit of a pain. Granted, they have fairly complete and flexible APIs but I can still imagine it being very annoying sometimes.

> I think the OP's point was that it forces you to do all the number crunching via those extensions [...]

Almost. If you're using C extensions for performance, those extensions can release the GIL, and then you get parallelism in CPython. Although if you've got a bunch of small steps, Amdahl's law is going to bite you as each step needs to re-acquire the the GIL to proceed.

Note that for Numpy, I don't know which functions/methods actually bother to release the GIL, but it's possible in theory.

Yes, that's exactly what's frustrating about Python.

It's not the performance per se, that doesn't really matter for Python's use cases, it's that the performance charateristics of your program are so erratic it doesn't feel like the pieces are fitting together. Denotationally equivalent operations having drastically diverging operational interpretations means that changing anything is a nightmare.

If you have an API call that does what you want, Python is basically C++. If you don't, you're paying a 100x penalty.

Most of the times performance doesn't matter, and when it doesn't, life is great. When it does matter, the situation feels worse than it actually is because optimizing Python is such a fight.

In general optimising Python is: * understanding that pure Python is not a language to write CPU-heavy computational code in it and was never intended as such * understanding that Python lets you write 3-4 solutions to a problem easy and 1 will be slow, two will be OK and one will be optimal - often that involves using generators, collections, sets and is a bit nuaced * I'd say that optimising Python, compared to something like C or C++ is far easier - I can set up a decent testing and profiling environment for most of my problems easy with pip and ipython which is borderline simple compared to valgrind

I'm not talking about technical difficulties, the Python dev experience is top notch and I have no complaints in that area. What I mean is that it's hard to form a mental model of what's going on because it's as though you were using two different languages: the "fast" API calls and the "slow" munging of the results.

The frustrating part is that the language doesn't seem to "compose" any longer, it doesn't feel uniform: you can't really do things the simple way without paying a hefty peformance penalty.

With that said, the #1 reason why any code in any language is slow is "you're using the wrong algorithm", either explicitly or implicitly (wrong data structure, wrong SQL query, wrong caching policy, etc.), and in that sense Python definitely helps you because it makes it easier (or at least less tedious) to use the correct algorithm.

I think that complaint applies to the (vast, essential) ecosystem, not to the language and standard library. AFAIK, there aren't serious performance potholes to hit in the language and standard library any more. You're not paying a huge performance penalty for writing things the simple way, but things are uniformly pretty slow.

It's when you pull in fast libraries that composition breaks down e.g. the many-fold difference between numpy's sum method and summing a numpy array with a python for-loop and an accumulator.

I agree, the standard library doesn't have those problems. As you said though, the ecosystem is kind of the point of using Python in the first place.

Again, I'm not really a fan of Python as a general purpose language for large projects, but it really, really excels at being a kind of "universal glue", and it's why I use it again and again.

In that role, "uniformly slow" is better than "performance rollercoaster".

spaCy might be a good example. It's a natural language processing library that's written in Cython, so yeah, a lot of the hard computation work is essentially just calling into C code.

But it's usually being driven from a Python script that's in charge of loading the data and feeding it into spaCy's processing pipeline. That code is subject to the GIL, and, since it cannot be effectively multithreaded, it tends to become the program's bottleneck.

So what you typically end up doing instead is using multiprocessing to parallelize. But that comes with its own costs. You end up wasting a lot of computrons on inter-process communication.

Could you provide a specific example where GIL prevents "complex data analysis"?

Though it's not only about number crunching. I tried to implement a streaming protocol for certain ip cameras: You may already see the effects of GIL + the general slowness when reading many small packets from a network socket in one thread and parse/process the packets in another. While Python is my favorite tool for many years and I was seemingly aware of it's downsides, I really didn't expect how bad it performed in this case.

> [...] when reading many small packets from a network socket [...]

Yeah, I should've said it more carefully. That's what I was hinting at with the mention of Amdahl's law. The separate bits of I/O are done in C extensions which can release the GIL, and those can be done in parallel (with each other and your computation), but if each bit of work is small, the synchronization cost (reacquiring the GIL for each packet) becomes relatively large.

I do care about the CPython GIL. I develop a program with a lot of analysis and output to custom formats. The things that can go to the underlying C extension go there, but the development cost is far higher for migrating stuff there. I'd like to be able to parallelize the different sections, but multiprocessing is a mess (only pickable data can be used, no shared state), threading doesn't give me any benefit due to the GIL, and PyPy hasn't worked the last three times I tried. Also, improvements can stack. If I get threading working, I can still move more things to the C extension to improve performance, or migrate finally to PyPy (if they manage to remove the GIL).

> but multiprocessing is a mess

Could you share a little more about your experiences with mp? I'm in the other camp - I use multiprocess in several python applications and am more or less happy with it. My real curiosity I suppose is what data you are trying to pass around that isn't a good candidate for mp.

My own use cases normally resemble something like multiple worker-style processes consuming their workloads from Queue objects, and emitting their results to another Queue object. Usually my initial process is responsible for consuming and aggregating those results, but in some cases the initial process does nothing but coordinate the worker and aggregator processes.

My main issue with multiprocessing is that the pickle module is incredibly unhelpful when something can't be picked. Given that there's quite a lot of context shared between types of analysis, and that context tends to change a lot, using pickle means I'm looking at quite some time debugging every now and then to see what's failing.

Regarding the data itself in this case, there are some objects that are references to in-memory structures of a C extension. Last time I checked I didn't see simple ways to share those structures via pickle. Shared memory was an option, but it required me to implement and change quite a lot of things just to get things working, not to mention that whenever the C extension gets new features I need to invest extra time in making those compatible with pickle. It wasn't worth it, specially knowing that I would still run into problems every now and then due to pickling the context.

> threading doesn't give me any benefit due to the GIL

If your heavy lifting is done in C extensions, and those extensions release the GIL before doing that work, you can still get the benefit of parallelism within CPython.

Part of the heavy lifting is in C extensions, but part is in Python code that's hard to migrate to C, and that's the part that takes more time now after migrating the low hanging fruit. I'd get a bit of benefit, but far from what I'd get if the GIL wasn't a thing.

Depending on the nature of the interpreted Python part of the code, you could get a 60-100 times speedup by using PyPy (no rewriting needed). Removing the GIL in CPython would only beat that if you could get 60-100 times parallelism.

I tried using PyPy several times, but I always run into problems due to the C extension and other libraries I use. Also, last time I checked it wasn't very up-to-date with respect to the mainstream Python version.

In the article, Victor brings up a good point that the GIL can lead to some nasty scheduling with IO. If you have some thread doing computationally expensive stuff and other threads responding to HTTP requests, the computationally intensive thread ends up hogging the CPU.

When a thread has been waiting for IO, you often want to switch to it as soon as the IO is finished in order to keep the system responsive. So the biggest issue with the GIL seems to be more about scheduling and less about actually having true concurrent threads in Python.

> I don't understand why people care about the CPython GIL

1. pypy has a GIL

2. not every python program is compatible with pypy.

3. not everyone can program in C.

4. it would be a free speed up for troves of multithreaded code.

that being said, the GIL is not going anywhere, there are enough workarounds and the task is monumental.

In ML you often need to stream data from the training set because you can't hold it all in memory. Many times I noticed that my GPU would sit idle because python was busy loading more training examples. With proper multi threading you can have one or multiple threads loading data into a queue feeds the GPU thread. This is something extremely simple to do in any programming language that has doesn't have a GIL. And you can't just use a library for this, loading and preprocesding data is very specific to your application, and writing just this part in C completely defeats the purpose of choosing python in the first place.

For number crunching it doesn’t matter, but I imagine it’s a much bigger issue if you want to run some kind of application server/webapp on a big beefy machine with lots of cores. That’s a thing you would want to run in something like Python, and the lack of parallelism is a real issue.

If your application is I/O bound (web app), then you can get parallelism, even with the GIL. Each I/O call can (should) release the GIL before it waits on it's network packets.

128 thread processors are already available at the prosumer level (hyperthreading is presumably a good use for python since it does so much waiting with pointer chasing, but even if not there will be 128 core consumer chips before long).

Reply on news.ycombinator.com