Hacker News Reader

Comment by mrfusion | Hacker News Reader

Can Python copy how other languages like golang or java operate without a Gil? Why or why not?

Python has different semantics to those languages. I don't think it's formally specified, but people program in Python expecting the semantics that reference counting provides, and that unsychrnoised concurrent access to data structures will not cause errors. Despite ongoing research, it appears to be hard to continue to provide these semantics without a GIL.

Golang (informally I think) and Java (more formally) are not specified to provide reference counting semantics and not specified to guarantee that unsychrnoised concurrent access to data structures will not cause errors.

So the languages have different semantics - that's why you can't copy-and-paste the solution from one to another.

Some alternative implementations of Python don't follow the above semantics, like Jython, but then some people aren't happy with that. It may not be acceptable to the community to drop those semantics, even if they were never formally given.

> and that unsychrnoised concurrent access to data structures will not cause errors

Python doesn't ensure that unsynchornized concurrent access to data structures won't cause errors. As I understand and experience Python multithreading, all the GIL ensures is that of the "load - inc - store" stages running amonst N threads, each separate stage will be locked, but not the overall sequence. So you'll still have data races, even with the GIL, and you still need to use mutexes etc in your Python program, which is why they are there.

I mean it won't cause errors within the basic data structure access operations. I'm not talking about composition. In Java a hash table write can fail with an exception if there is a concurrent write that conflicts. A Python dict write is atomic, because it happens within a single instruction as you say and so will not be interrupted and will never fail. That's what you aren't getting in Java. That expectation is very hard to provide without a GIL. Jython does it with blunt fine grained locking, but that's slow.

Interesting food for thought. So what exactly do you lose in jython?

You loose access to all the C-libraries that comes with Python. On the other hand you get beautiful integration with all Java libraries very nicely integrated.

As the talk mentions, Jython (Python on the JVM) is living proof that it's possible: it doesn't have a GIL and it works just fine.

The issue is "just" one of implementation. Sun and Oracle have spent millions of dollars paying ridiculously-smart people to make HotSpot as good as it is... while the CPython codebase has traditionally aimed at being simple enough for average C programmers to contribute to. I'll be interested to watch how that tension plays out.

How much existing Python code works with Jython? Particularly that depends on C extensions like numpy?

Both Jython (Java impl) and IronPython (C# impl) works fine without the GIL. The big issue is compatibility with C-based libraries, as I hear Larry Hastings, there are three levels of compatibility:

1) Fully compatible, nothing needs to be done 2) Fully compatible, but a recompile of C-libs is needed 3) Almost compatible, but some updates to C-libs are needed

Larry is aiming for 2)

Yes, it can. The purpose of the project is to figure out how to do that while minimizing the number of APIs that have to break.

Reply on news.ycombinator.com