Hacker News Reader

Comment by MostAwesomeDude | Hacker News Reader

Disagree; Python's hardest problem is packaging, followed closely by bikeshedding over networking libraries and an ever-growing dichotomy between the Python 3 and Python 2 universes of code.

You can skip all conversation about the GIL and threads neatly by simply preferring a different concurrency model. There are plenty of ways to do this (see above re: bikesheds and their colors), but being permanently tied to CPython and the threading module is increasingly uncommon for professional Python, and it isn't as unavoidable as things like networking or even which language you're going to use.

Edit: I see that the author's in this thread. Nicely written article, but a tad hyperbolic.

View on HN

There isn't a growing dichotomy, there is a migration from 2 to 3. We have passed critical library mass I feel in Python3 space and now it is just waiting for the world to catch up and just use 3 already.

The transition was always planned to take several years and I can't think of any major project which decided not to port to python 3. http://py3ksupport.appspot.com/ and https://python3wos.appspot.com/ both list package compatibility.

What issues do you see with python2 to 3 transition?

To be fair, the Python 3 transition timer shouldn't have had its tick starting until more recent versions of Py3. This won't be true for all Python communities (eg. Mathematics / Research), but until the more sane implementation of bytes/strings/unicode, the transition for most web frameworks or protocols was a mess.

However, from the later list, there are some really damning packages. For one, we're going to have transition off Twistd, which is much more than just an event loop. Py3 might not have enough advantage(s) to compel us to make that transition.

http://twistedmatrix.com/trac/milestone/Python-3.x

So, just wait to migrate until twisted is ready?

Honestly, I didn't check the dozen-plus packages pending update for us and then someone else cutting their teeth on it for a while, but Twisted was clearly marked as not being updated for Py3. But yes, clearly we're waiting; apparently Guido also wants to take a stab at the async batteries, which could help or hurt, if the Twisted contributors think they will be affected.

To the point of the comment you first responded to, there is lot of issues not being "solved" with respect to network libs, their upgrading to Py3, and Py3 looking to step into that arena. In other words, there is no horizon for networking libs updating, and they may feel even less motivated if the Py3 crowd is saying, "don't worry about it, because we are going to standardize our way." Does the Py3 transition-clock only start then?

I'm not really sure what your issue is. You use twisted. Twisted is currently being updated to support python3. Gevent has a branch which passes tests for python 3.3: https://travis-ci.org/fantix/gevent/builds/3588293 Is there some major library you are using that has no plans for python 3?

I'm not actually seeing an issue with python 2 -> 3 transition which is what my original comment was asking about. I see a lot of negativity surrounding python 3 and talks of fracture, but no actual evidence of it.

> I can't think of any major project which decided not to port to python 3

Most major projects have not actually switched to Python 3, that just support it using 2to3.

Actually, the trend is away from using 2to3 - it's now seen as preferable (and quite possible) to have one codebase which works on both 2 and 3. That's the route Django went for, and what Flask is doing. Numpy also switched from using 2to3 to supporting both from a common codebase.

So restricting yourself to a common subset of Python 2/3 and adding compatibility helpers in your code is the preferred approach now a days?

I'm still not convinced about the whole Python 2 to 3 conversion being worth it (asides from the fact that not porting means your project is seen as not being actively maintained). It has led to added complexity in code so that it works on both versions, increased difficulty in packaging/distribution, having to test/debug on both versions, obsolescencing vast amounts of code that will never be ported, confusion among new users, untold man-hours being spent, ...

I don't mean to be sound so down on Python 3, but it has caused me nothing but additional work and frustration. The supposed benefits of Python 3 are still years away.

The common subset is most of the language, if you target 2.7 and 3.3. But yes, it has clearly caused extra work. But all the tools I use support Python 3, and I'm looking forward to when new users routinely go for 3.

Absolutely agree.... in other words the community. To that I would add the ever growing dichotomy and fragmentation regarding improving performance between people solving this (CPython vs. PyPy vs. Continuum). I'd trade a python that was 5-10X faster (and could use numpy/scipy) over one that didn't have a GIL.

I don't think it's quite accurate to frame the current state of innovation in High-Performance Python as being one in which the principal actors are fundamentally opposed to each other. Nothing that we do at Continuum detracts from the CPython ecosystem; in fact, our tools are designed to specifically provide maximal utility, as quickly as possible, to certain (significant) groups of Python users. We're not tackling anything nearly as ambitious as PyPy's meta-tracing JIT or generic STM. Much of our work builds on tools and lessons learned over the last decade, including work on Cython, Mython, LLVMpy, etc. etc.

As for the GIL, the OP is right that it's a CPython implementation problem, not a structural one that has to exist in Python. There is some exciting work that has been done in the direction of fixing this at the implementation level, and we are excited about seeing how far we can take it. Stay tuned...

Right. If you're spending a lot of time in the CPython interpreter, you probably don't care too much about performance anyway. The CPython interpreter is a hell of a lot slower than using Numba or Cython (which can operate without the GIL) or PyPy (which is an completely separate codebase from CPython, although it still has a GIL in its typical incarnation), or many alternative languages. As soon as you're out of the interpreter, the GIL ceases to be an issue (e.g. NumPy releases the GIL while performing array operations).

I've enjoyed the Python networking libraries I've used. A lot. Could you be more specific?

Also, I'm not quite clear on your issue with packaging. Can you be more specific? Compared to C, I find importing existing code to be nearly heavenly.

Regarding backwards compatibility, maybe it's an issue, maybe it's not. On the one hand it's frustrating and some people will make it an issue. On the other hand, I can't help but think of cases like Apple, where they were the first to abandon modems, serial ports, diskette drives, and CD drives—but probably made the right decision.

Hes not talking about Python imports, hes talking about pulling Python packages through pip or one of the dozen utilities to do so from pypi. Though I do feel pip has been standardized on enough to solve that problem.

There are two options for Python networking: Twisted, and things that are not Twisted. Guess which one most people pick.

What do you use to package? distutils, setuptools, distribute, distutils2, pkgutils...? How to choose?

start with distutils which is in stdlib and if it doesn't provide enough features; use setuptools (distribute and setuptools are merged) that extends distutils functionality.

Packaging is fine unless you're ignoring the current state of tooling. Not sure what you mean by networking unless you're talking about gevent/twisted/tornado/diesel which are really proxies for the problem of concurrency (GIL) rather than expressly about networking.

Packaging is starting to come together, but still has a ways to go. The Setuptools-Distribute merge was just announced a couple months ago, we still have 6 ways of doing things that are mostly incompatible. Packaging is mostly solved if you're on Linux, sort of solved if you're on OS X, and a horrible mess if you're on Windows. Perhaps if you can live in the world of pure Python it's good, but once you have C-based dependencies, it quickly becomes a mess. Compared to something like CRAN and the R community, it's clear Python could be improved in this area.

> bikeshedding

Not just network libraries; How many cli modules are there now?

Reply on news.ycombinator.com