Hacker News Reader

Comment by blibble | Hacker News Reader

indeed

not sure there's much they can do about this, other than protecting all the built-in data structure operations with mutexes, like java's original data structures (Hashtable, Vector, etc)

(but then how do you get a non-synchronized [] if you want one?)

View on HN

My impression was that that was exactly what they were going to do: replace the GIL with fine-grained locking on the objects themselves. I can't imagine they'd let multiple python threads manipulate python data-structures concurrently, the interpreter would segfault immediately.

> (but then how do you get a non-synchronized [] if you want one?)

You don't. This is one of the reasons why using the GIL is higher performance for single-threaded use-cases: stuff like lists and dicts can be non-synchronized

> not sure there's much they can do about this, other than protecting all the built-in data structure operations with mutexes, like java's original data structures (Hashtable, Vector, etc)

However - this is fundamentally the incorrect approach, because Vector and Hashtable aren't protected from read-then-write race conditions.

Such internal locking guarantees that the collection stays structurally sound, but not that code accessing it is dealing with a single consistent state until it finishes.

I think you can make a non thread-safe list by just making the same object without the lock described at https://peps.python.org/pep-0703/#container-thread-safety if you really want the maximum performance of single threads.

Maybe it could be part of a non_threadsafe_containers module on Pypi.

You could fast path it by checking if the reference count of the list is `1` and avoid taking the lock in that case, I think.

Could you? What enforces only a single thread having access to a given reference? What about global variables?

If there's a refcount of 1 you can mutate the value safely because no other thread could be trying to read/ write to it. And the only thread that can give it to others would be the one that's doing that mutation, so it can't suddenly change.

I'd assume that importing any sort of module level variable would imply an increment of the counter, but unsure.

that would work on x86 but not on an arch that can re-order loads (e.g. arm)

I'm assuming it would be a fenced operation

but then there's still an advantage to using the fenceless version :)

(admittedly it's python so it's so slow it's probably not even measurable)

Yeah, for Python I feel like the difference between fenced vs unfenced doesn't matter. The primary cost is around your L3 cache getting slammed with contentious atomics but your L3 is already absolutely fucked if you're using Python.

Reply on news.ycombinator.com