Hacker News Reader

Comment by saurik | Hacker News Reader

It continues to be dissapointing that new releases of Python continue to be useless due to the (I will claim needless and arbitrary, although I could totally argue the other side if forced: I am not ignorant of the reasons, I just disagree with the opinions) decision to make the new builds incompatible with the previous ones (and in some cases for the obvious worse, such as how Unicode is now less correctly handled when iterating filenames in directories).

I mean: I do almost all of my "business logic" coding in Python these days, and yet when I hear of a new version of Python it carries as much weight to me as a new version of COBOL (which, honestly, probably has more use cases, and almost certainly more users, than Python 3.x). This is the first time I have simply totally felt stranded by a development environment since Microsoft abandoned Visual Basic and replaced it with "Visual Fred" (aka, Visual Basic .NET): and in that case I had already jumped ship to C++ years earlier.

At this point, the cost of trying to get all the libraries I rely on set up on 3.x--especially when a ton of them have decided either 1) to slightly change the semantics of the API "as everything is breaking anyway" or 2) that they are obsolete and that you should use some other newer library if you want 3.x--is sufficiently high that one may as well switch to an entirely different language (in my case, probably Clojure).

View on HN

> It continues to be dissapointing that new releases of Python continue to be useless

Useless for you :). Personally, I made the switch to Python 3 some time ago, and couldn't be happier. My dependencies are largely sorted these days, availability on the systems I need to deploy on is no longer an issue either, and I find it to be a cleaner, leaner language makes me more productive overall.

Sure, there was a time when I read these 3.x announcements with a sort of weary, frustrated feeling, but now I'm back to feeling like a kid in a candy store and excited at what the new release is going to bring.

I realize not everybody has made it to this side of the transition yet, but don't fret it, the day will come, and it's a nice place to be in.

As I explained, for me that day will only come when I switch out a ton of libraries whose bugs and warts I had to painfully work around, as the maintainers have either left or explicitly replaced with different libraries. I question whether this effort is worth it.

I am curious, however, how many libraries you actually rely on, given the ease you seem to indicate you were able to move with. Most of my code is just gluing together libraries, from wrappers for libapt to PDF parsing and generation (for which I actually have to use two different libraries) to database bindings to advanced math. I have pages of import statements in my codebase...

Near as I can tell, were I to start fresh and use Python 3, even if all the libraries I currently use were ported, I'd just come across a new use case that causes me to need a new library, and as that library will likely not be ported, I'll just be cursing my decision to have switched. Seriously: switching from Java to Clojure has less potential future regret than switching from Python 2 to 3. Hell: Java to C++ has better backwards compatibility (JNI is actually reasonable, and is easily abstractable).

> I am curious, however, how many libraries you actually rely on, given the ease you seem to indicate you were able to move with.

Not that many, admittedly. I'm fairly wary of dependencies and prefer to stick to the standard lib unless something is just orders of magnitude better at the job, to cut down on deployment overhead. That attitude has of course benefited me tremendously with the Python 3 transition.

That said, I currently use lxml and requests a lot, a long with a bunch of smaller things like python-dateutil and mock (the latter of which was just added to the standard lib in 3.3, actually). Sum total I have maybe 12-15 external libs across my codebases that I didn't write myself, and I've had only one case of an abandoned lib that I needed to port myself.

As for my own code, when I started out transitioning I had already cut compatibility with anything older than 2.6, and making things run both on 2.6 and 3.1 turned out to be relatively easy for the most part (certainly with some pain points too however, like needing to abandon syntactic sugar for metaclasses). As the next step I went to 2.7/3.2, which made things even easier. Most of my code now still runs on 2.7 in addition to 3.2, but I'm now starting to abandon the notion of backwards compatibility and introducing code that relies on Python 3.2 and am slowly removing the hacks once required for cross-compat as I find need to touch the respective areas again.

Now, I realize that I'm not the norm and that a lot of Python folks out there have a much harder time transitioning. But with the bleakness of your post, I felt compelled to point out that it's not been hell for all of us.

> As I explained, for me that day will only come when I switch out a ton of libraries whose bugs and warts I had to painfully work around, as the maintainers have either left or explicitly replaced with different libraries. I question whether this effort is worth it.

I can't see myself dropping 2.x support anytime soon.

I respect your opinion and maybe you’re in a unique position that makes Python 3 really awful to you, but in general your comment seems more like a knee-jerk "lolpython3" reaction that was appropriate while 3.0 & 3.1. But now, that situation changed.

Many, many high profile projects are already on Python and many will follow, now porting has been made even easier.

Django tries to support Python 3 with 1.5, Pyramid already does support it, Twisted has a GSoC student that tries to bring it to Py3… And funnily enough, “normal” code that doesn’t screw around too much with advanced features is rather trivial to port. I have written a Python 3.2 program myself only to find out later that it also works on Python 2.7 without changes. :)

With every bigger change you lose some – and I’m sorry if it’s inevitably you – but you also win some. Because all in all, Python 3 is great.

There's a bunch of developments that make me feel that the Python 3 ball has started rolling in a much more significant way lately. Some you've already mentioned; another is Canonical insisting on everything in the "main" section of Ubuntu running on Python 3 by 2014, and doing some of the work to make that happen.

> another is Canonical insisting on everything in the "main" section of Ubuntu running on Python 3 by 2014, and doing some of the work to make that happen.

Everything on the install CD is planned to be Python 3 by this fall. I'm already working to port what's necessary for Ubuntu One.

I've noticed that the top 3 most requested packages to port now all have Python 3 coming in a release soon. (Django, matplotlib and wxPython - from the poll on the python.org homepage)

I take severe issue with your characterization of my comment as both a unique and knee-jerk opinion. I will even go so far as to say that your comment is needlessly patronizing, and thereby horribly insulting.

In my comments, I describe a specific phenomena that are leading to the problems that I experience: that there is now a fractured community of libraries for what are really two unrelated languages.

As some even-more-specific actual-examples:

1) key libraries such as PIL are not ported, despite your claims regarding the ease

2) of those that are, such as psycopg2, there are often awkward (and I will often claim "broken") semantic changes to the behavior of the API

3) not all libraries actually have maintainers to port the code at all (such as the iPhone PNG transcoder I rely on)

4) finally, there are libraries, even entire frameworks (mod_python), that rely on functionality that is actively being removed with prejudice (in the case of mod_python, for reasons I continue to not understand, despite having read copious e-mail exchanges on the Python mailing list regarding the issues)

(A quick aside, for anyone about to pull a "lol mod_python" on me: mod_python is fundamentally more powerful than the various replacements, something even the primary maintainer of mod_wsgi admits; there is a reason why, despite it having been retired from Apache due to a lack of maintainer, it was still used by Google engineers as the basis of their pywebsocket implementation.)

I will therefore challenge you to put up something behind your insults, and explain what you feel has changed, and how it affects the argument I have made. While it is of academic interest that a few web frameworks now support Python 3, real programs, even real websites, are going to rely on libraries to accomplish their underlying goals.

I will even further challenge you to defend your accusation that my experience is "unique"; I personally find this idea incredible, given that I ask every Python programmer I meet "2 or 3", and not a single person has ever said "3". Yes: if you call out Python 3 in public on a Python 3 release post, a number of people will come out of the woodwork, but in a random sample under more realistic conditions (non-Python developer conferences) the reality seems closer to 0%.

In my comments, I describe a specific phenomena that are leading to the problems that I experience: that there is now a fractured community of libraries for what are really two unrelated languages.

As some even-more-specific actual-examples:

1) key libraries such as PIL are not ported, despite your claims regarding the ease

2) of those that are, such as psycopg2, there are often awkward (and I will often claim "broken") semantic changes to the behavior of the API

3) not all libraries actually have maintainers to port the code at all (such as the iPhone PNG transcoder I rely on)

I will even further challenge you to defend your accusation that my experience is "unique"; I personally find this idea incredible, given that I ask every Python programmer I meet "2 or 3", and not a single person has ever said "3". Yes: if you call out Python 3 in public on a Python 3 release post, a number of people will come out of the woodwork, but in a more random sample under more realistic conditions (non-Python developer conferences) the reality seems closer to 0%. Hell, even the comments I've made on this thread have been upvoted quite a bit (+22 on my root argument).

> It continues to be dissapointing that new releases of Python continue to be useless due to the (I will claim needless and arbitrary, although I could totally argue the other side if forced: I am not ignorant of the reasons, I just disagree with the opinions) decision to make the new builds incompatible with the previous ones

FWIW, and while I disagree with you wrt long-term (though short term I'm still on Python 2 due to working on a non-ported 10 years old code-base with dependencies on non-yet-ported libraries), 3.3 has added some concessions back for ease of porting with the re-introduction of the unicode literal prefix `u` (see PEP 414).

> and in some cases for the obvious worse, such as how Unicode is now less correctly handled when iterating filenames in directories

Either it's handled or it's not, Python 3 provides both options for you to pick, linux filesystems are broken either way.

> in my case, probably Clojure

From my playing with it so far, even if I switch "big" developments to that Python will have to remain part of my toolbox due to how horrendously slow Clojure and clojure-based tools are to start up. And I have issues with the utterly terrible error reporting as well.

> FWIW, and while I disagree with you wrt long-term (though short term I'm still on Python 2 due to working on a non-ported 10 years old code-base with dependencies on non-yet-ported libraries)

Just how "long term" are you thinking here? Another four years? Does it really seem likely that this will change? At some point the long term horizon is so far in the future that it becomes meaningless.

As an example, will the entire set of libraries you are using even have maintainers anymore? I will argue by the time Python 3 even has a hope of satisfying this bootstrapping problem, they will have moved on.

> Python will have to remain part of my toolbox due to how horrendously slow Clojure and clojure-based tools are to start up

Yes: that is currently the thing that is most bothering me about Clojure. That said, I have some plans for how I can fix that, and those plans, as ludicrously complex as they are, honestly sound easier to me than the Python 3 porting problem.

(In fact, modifying Python 3 to run Python 2 code sounds easier to me than the transitive-to-all-libraries porting problem, and has therefore been on my todo list. However, I have the problem that I need to get work done immediately...)

Regardless, I have little choice here: the Python 2 ecosystem is dying by fiat, while the Python 3 ecosystem has no real impetus behind it. The result is that the entire Python ecosystem seems to be stagnating: it is harder and harder to find good libraries for new things.

>From my playing with it so far, even if I switch "big" developments to that Python will have to remain part of my toolbox due to how horrendously slow Clojure and clojure-based tools are to start up

While I agree that tools which take a long time to start up can be annoying I don't believe that an IDE which takes 30 seconds to start up, but is left open for days at a time should be an issue. Are you referring to the IDE or the interpreter?

> Are you referring to the IDE or the interpreter?

Interpreter and command-line tools (e.g. leiningen)

Amen. Python 3 has created an incredible amount of work in the aggregate for the authors of all the 2.X libraries that must be ported. Even worse, libraries that are no longer actively maintained must now be taken over by someone new and ported. All for new language features of very dubious value. If Python 3 brought us real threading with no GIL, that would be worth the pain. But instead, we get removal of a few warts and backward-incompatibility. One can only imagine how much damage the years of incompatibility and confusion caused by this decision have curtailed the popularity of the language. 2to3 is impressive and useful for implementating arbitrary code transformations, but obviously it can't magically make old code handle bytes/chars/unicode properly from a semantic standpoint.

And think about how the 3.X changes totally screwed the Jython and PyPy (and Cython, and IronPython, and...) teams. We were headed towards much higher-performance implementations of the language, with additional powerful features like sandboxing and JIT, and it was set back years by the 3.X schism. Furthermore, the continued complication of the language makes it ever less likely that core problems like the GIL will ever be fixed. (Yes, I know Jython and IronPython are GIL-free.)

My startup relies heavily on Python, and the only reason I don't seriously regret choosing Python as a core technology is that there was nothing else better at the time. The Python overlords have reduced the language to the best of bad options; it didn't have to be this way. FAIL.

I still hope against hope that there will be a Python 4 that brings backward compatibility and GIL removal. But since none of the overlords seem to care about either of these goals, and since both get harder with every new Python 3 release, it's probably just my own sad little fantasy.

> And think about how the 3.X changes totally screwed the Jython and PyPy (and Cython, and IronPython, and...) teams. We were headed towards much higher-performance implementations of the language, with additional powerful features like sandboxing and JIT, and it was set back years by the 3.X schism.

I don't see how the 3.x changes have set back PyPy "by years". PyPy was focussing exclusively on 2.x until very recently, and made its last big steps towards production-readyness entirely undisturbed by the 3.x transition. As of a couple of months ago they now have a small team working on 3.x support, which has made huge progress in a short time frame and doesn't seem to have detracted from progress in the core and further 2.x release at all as far as I can see.

> Furthermore, the continued complication of the language makes it ever less likely that core problems like the GIL will ever be fixed.

Ironically, specifically the oh-so-handicapped PyPy is currently working on a new approach to address the GIL with their transactional memory work. And the language has gotten less complicated with 3.x, not more.

> I still hope against hope that there will be a Python 4 that brings backward compatibility and GIL removal. But since none of the overlords seem to care about either of these goals [...]

Right, that's why 3.3 just added back support for u'' for increased backwards compat and ease of porting.

You're making a lot claims that simply seem to have no basis in reality.

The reality is self-evident from the PyPy home page:

"PyPy is a fast, compliant alternative implementation of the Python language (2.7.2)."

Why not 3.3? Oh, right, because when the PyPy team started, Python 2.X was the Python language. Now it's not. This has been the case (PyPy has existed and supported 2.X code) since 3.0 was released. I.e., for years.

As for PyPy STM, you trot that out like it's a real solution. It is not. It's 5 times slower and doesn't support Python 3.X. Which, of course, is the whole point.

But, yes! Thank goodness I can type u'...' again! That will save me years of work! FAIL.

> As for PyPy STM, you trot that out like it's a real solution. It is not. It's 5 times slower and doesn't support Python 3.X.

This sort of comment doesn't really make me feel like you're qualified to comment on the subtleties of killing the GIL, to be honest.

When it comes to removing the GIL, any form of "it's slower" is usually going to refer to single-threaded workloads, and avoiding to be slower for those is going to be nigh on impossible. The idea is that it's an investment to be recouped for various workloads due to the improved ability to exploit concurrency over a GIL-ful interpreter.

And really, it's at the proof of concept stage right now, hardly optimized. And there's new CPU hardware with hardware support for transactional memory coming down the pipe.

> The idea is that it's an investment to be recouped for various workloads due to the improved ability to exploit concurrency over a GIL-ful interpreter.

This idea has been rejected by the core team though, getting rid of the gil with a major performance hit is not considered acceptable when most existing Python code is not significantly affected by the GIL.

Aye, which I think is reasonable, especially in the context of the GIL-removal approaches that were on the table at the time and the effort needed to implement them in CPython.

I think the PyPy folks want to get to a place where you can run PyPy in modes tuned for either single-threaded or concurrent workloads.

You are making excuses. It should require neither new hardware nor Herculean effort to realize acceptable multicore performance on CPU-bound tasks. And your badgering and down voting people here who disagree with you position doesn't make your argument any more persuasive.

Just to come to your aid and make your last sentence not a baseless accusation: Yes, I've been downvoting you. However, it's not because I'm disagreeing with your position, it's because I consider the way you conduct yourself in the debate, well, crappy, because:

- First and foremost, your frequent use of ridicule and hyperbole is escalating the tone of the debate.

- You prefer to move goal posts instead of addressing arguments (e.g. you still haven't actually gotten around to explaining how the 3.x transition has supposedly set back PyPy by years).

- You're not beneath skewing things to fit your narrative, even if it doesn't really make any sense (e.g. pointing out that PyPy's STM tinkering doesn't yet support Python 3.x yet - yeah, duh, given that both are WIP branches that aren't merged into PyPy proper yet, and nevermind that the STM work doesn't really impact the same area of the codebase that adding Python 3.x support does, so there is no cause to anticipate problems with eventually integrating both).

- You make bold claims and leave them unsubstantiated (e.g. "It should require neither new hardware nor Herculean effort to realize acceptable multicore performance on CPU-bound tasks." just now - if you've got it all figured out, where's your proposal to eliminate the GIL?).

- Those bold claims even miss their mark in the context of the debate (I pointed out that "it's slower" refers to single-threaded workloads; when it comes to multicore performance the STM proof of concept is already showing some benefits).

And so on and so on. Simply put, your posts are low quality, that's why I'm hitting that downward arrow.

I still don't see that you are actually challenging the original claims, which were simply that splitting the language in two was a very bad idea, and that the split damaged the Python ecosystem.

I stand by my claim that any reasonable general-purpose programming language should be able to run simple CPU-bound threaded tasks properly. I didn't claim arbitrary parallel programming should be easy; just that the GIL causes multi-threaded programs written in standard style in Python to not work properly in CPU-bound contexts, because the GIL prevents these programs from using all the cores available, and, furthermore, because even in single-core contexts it is prone to performance failure modes such as convoying and starvation that induce worse performance when there are more cores available. (cf. very extensive discussions in bugs.python.org on GIL problems, along with David Beazley's, Antoine Pitrou's, and others' attempts to address them).

It is a standard, uncontroversial expectation of general purpose languages that offer threading that the threads run in parallel, using the available cores. I don't believe this needs substantiation any more than the claim that general purpose programming languages should run serial programs deterministically.

I believe the better course for Python post 2.X would have been to fix the core problems with the language's reference implementation, while maintaining backward compatibility with the very large existing code base, rather than adding features of dubious value, doing nothing to address the GIL problem, and breaking compatibility with the vast majority of 2.X code. While this is certainly my opinion, and yours obviously differs, you can't badger away the limitations of the GIL, and you can't just browbeat people into accepting the 3.X series as a good thing.

Look, I don't hate the language -- I use it every day and like it a lot. But don't pretend that there aren't real problems with Python when there clearly are -- the discussions on bugs.python.org are mainly driven by strong proponents of the language and are hardly politicized -- and don't pretend that Python 3 wasn't problematic; it's created years -- yes, years -- of problems and confusion with 2 vs 3 incompatibility, has expanded the footprint of the language and changed its semantics in substantive ways (cf. the many, many accepted PEPs post 2.X) that make implementing the language harder. Every language bump (whether 2.5 to 2.6 or 2.7 to 3.0) has meant additional work for the developers of alternative implementations; this is obvious on the face of it; you need only read these projects' blogs and look at the elapsed calendar time.

You seem to presume that if there weren’t Python 3, we’d be busy removing the GIL. That is not the case, removing the GIL has no priority for reasons already too often discussed.

No, I neither believe nor tried to assert that. I said I believe that effort should have been invested in fixing the GIL and other core language problems rather than adding more stuff to the language, fixing a few warts (and, admittedly, one big problem, chr vs unicode vs byte).

I also claimed that the additional features added to Python 3.X have meant that those who have already fixed the GIL problem (Jython/IronPython) and those who might still (PyPy) have been forced by the 3.X change to chase the taillights of CPython, making it essentially impossible for these implementations to replace CPython as the canonical version.

(I do take for granted that the canonical implementation of Python should have multithreading primitives that can take advantage of all the cores in machine; that has been debated above already so I won't repeat it here.)

Ironically, Py3s purpose _is_ to fix Python’s warts. You just seem to be lucky enough to not have encountered them yet. :) Python 3 was about _losing_ baggage so we can start adding features again.

The fact that Python 3.3 has a lot of more features than Python 2.7 is is due to the fact that 2.7 was the last release that got Python 3 features back ported (two years ago) and is in bug-fixing mode since then. (and people in these comments still complain about Python moving too fast)

There also have been efforts to ease the pain for PyPy et al by requiring new modules to have also a Python based implementation and by – put simply – giving them money from PSF funds. Jython’s problem was rather lack of interest and developing funds as they were stuck at 2.5 forever until one of their developers got funding to bring it to 2.7.

It’s not all as bleak as it seems to you and chances are that your porting effort will be negligible as soon as the dependencies are met.

And the GIL…just leave it be. There are really no plans or efforts to remove it for many, many reasons I won’t reiterate here.

OK, it's now off-topic, but nobody is probably reading this thread anymore anyway and I can't resist posting some sort of epitaph here.

You're saying it's not bleak, but then immediately say the GIL is a permanent fixture of the language. The GIL not being fixed is in fact what makes it so bleak (to me, at least).

Python isn't just for scripts anymore; we have 150k lines of Python code doing nontrivial stuff and relying on many third party libraries -- ultimately probably a million lines of code or so if you trace the dependencies all the way down.

This is in most ways really great: the Python code is very clear and concise, and comes along with all the documentation and unit testing conventions established by the community over the years. All those high-quality third party libraries mean we don't even have to write lots of code in the first place. It's easily a 10x productivity improvement over C/C++ for us.

But to make this code base take proper advantage of multiple cores, we must run under Jython or IronPython. (And at least Jython has its own problems; start up times are long because the Java class loader is slow.) We can't move to 3.X even if we wanted to, because no GIL-free implementation supports 3.X. We can't rely on the processing module, because we can't afford the overhead of an entire process of memory usage per "thread" at scale.

It just sounds like the Python devs have decided that fixing the GIL is too hard, so people should just live without using multiple cores. Do they view our use case as exotic or somehow unsupported? Do they not see that it's important for a general purpose programming to support real threading? I'm not trying to be obnoxious; it's a serious question.

We are likely to ultimately abandon the language because of this core issue, which is tragic given the other great things we get from using it, and the lack of a genuinely equivalent alternative. "So what, no one cares about you -- you are nobody," you may say. For the sake of argument, I will stipulate that. But I don't imagine our constraints are unique, and this bodes ill for the Python ecosystem. Our abandoning it is one thing; increasingly many developers abandoning it as multicore use becomes ever more important is another. Cores are getting a little bit faster with each tick and tock at Intel. But the rate at which we're getting additional cores looks to far outpace the per-core improvements for quite some time. This makes real threading increasingly important as time goes on.

I know STM is the solution du jour, and it's an admirable piece of work. If that solves the problem in the long run, great, but it's far from obvious that it's going to win without dedicated hardware support, and in any case isn't part of the reference implementation of the language (which seems to always be ahead of the other implementations).

I know you're saying it's not a big deal, don't worry about it, etc. but it just doesn't seem that way to me. :(

The general opinion is that you shouldn’t use threads unless really necessary. Processes or eventlets are usually preferable. Python is not the only high-level language with this "problem" btw.

FWIW, my personal opinion is that the future of threading in Python lies clearly in PyPy and STM. Their performance is nuts and the contributors are smarter than both of us. At some point the STM overhead simply won’t matter.

I actually agree that it damaged the Python ecosystem, at least in the short term. It certainly did have a negative effect on Python momentum that will be felt for some time to come. I believe the Python developers anticipated this, struggled with it, but did it anyway because they believed it to be the right call in the long run. I'm hoping they will be right, because I do enjoy programming in Python 3 more than I did programming in 2.

Would it have been a better call to work on problems like the GIL instead? Perhaps. Certainly many, yourself included, would consider that a more significant problem to solve than the comparatively minor touchups that Python 3 did to the language. However, I do tend to disagree there: I think that the slow accretion of cruft that bogs down a language over time is a more significant problem and more likely to cause a language to fall out of favor. And I think the timing works out in such a way that Python can recover the lost momentum from the transition in time for projects like PyPy to come up with a good answer to the GIL.

So, yeah, we disagree ultimately, but thank you for explaining your POV better. Upvoted.

> Every language bump (whether 2.5 to 2.6 or 2.7 to 3.0) has meant additional work for the developers of alternative implementations [...]

This is true, but I'd say the core Python developers haven't been entirely blind to this concern, cf. the 2 year moratorium.

Huh? It's a pretty widely accepted fact that correct & performant parallel programming is inherently a hard problem (barring invention of a silver bullet...)

Indeed, and I am not claiming otherwise. However, it is well understood (see bugs.python.org and search for "GIL") that the GIL causes performance problems in otherwise correct and performant Python code. For example, such programs can run more slowly if there are more cores available than if there is only a single core available. Various workarounds to the GIL implementation in CPython have been proposed to address these problems, and it's better in Python 3.X, but the problems still exist.

And one still cannot write true multithreaded programs in Python using the language's standard thread library. To use all the cores, you have to resort to the processing module, which relies on processes rather than threads; this vastly increases the memory consumption, and makes the resulting programs less idiomatic as well. (You have to pass only pickle-able stuff to your child processes, for example.)

I was just taking issue at your more general claim about parallel programming.

As for the GIL, it's such a dead horse. Threads are a very problematic parallel programming model and parallel threads would bring in an enormous amount of implementation complexity as well, if it even could be pulled off.

If your code and environment works, why are you changing it?

If you're updating just to update, then the problem isn't with Python...

Funny, I remember HN railing on a guy just a few weeks back for bitching about a bug not being fixed in an older version of perl and having the gall to complain to the perl devs about it.

One man's stable environment is another man's technical debt I guess...

I believe I explained that: because this isn't a static system, and I thereby need to continue to have access to new libraries that work with new external systems. As for your second sentence, that was needlessly insulting.

A portion of our codebase is 2.x and growing. Unfortunately, until we get to a point where our users uniformly have 3.x and no 2.x on our systems, we're going to be 2.x.

Each time we've checked out 3.x, we're like, "meh".

Reply on news.ycombinator.com