Hacker News Reader

Comment by efoto | Hacker News Reader

"The biggest source of problems might be multi-threaded programs with concurrency-related bugs that have been masked by the GIL until now."

View on HN

> concurrency-related bugs that have been masked by the GIL

Yeah... could phrase this as "All programs written with the assumption of a GIL are now broken" instead. Wish they had done this as part of the breaking changes for python 3, I guess they'll have to wait for Python 4 for this?

GIL can remain default on. Users can simply disable it for chosen parts of their program.

No they can’t, it would have to be disabled or enabled globally at the process and/or interpreter level.

I think I read that there won’t be another big jump like there was for Python 2-3. If I understood correctly, there could be a Python 4, but it won’t indicate huge breaking changes, it’ll just be another release

Yes. I once discovered that CPickle was not thread-safe. The response was that much of the library didn't really work in multi-threaded programs.

You mean programs where you put an object into pickle and some other threads modify it while pickle is processing it? Doesn't surprise me - the equivalent written in plain Python would be very thread unsafe as well.

No, I mean several threads doing completely separate CPickle streams with no shared data or variables at the Python level.

Has it since been fixed?

Probably not. CPickle is famously shunned by anyone who has to do serious, performance-critical serialization/deserialization.

I was curious, and an issue that fits the description was fixed in Py 3.7.x here: https://bugs.python.org/issue34572 but other threading bugs remain: https://bugs.python.org/issue38884

What’s the most performance critical alternative? Pickle is tied to the VM, so it’s not a generally good persistence option in a prod setup, but it can mighty convenient.

What do people use instead?

JSON, protocol buffers, thrift, etc. Saving python-native objects such as functions, class instances etc is usually not the right thing to do in production code.

Reply on news.ycombinator.com