Hacker News Reader

Comment by semiotagonal | Hacker News Reader

I wonder how many people feel that migrating to Python3 would have been worth doing in the absence of being forced to do so.

Dropbox invested three years of work, actually hired Python's creator, and are still not done. What are they getting out of it that they wouldn't have gotten if Python2 simply had been maintained?

View on HN

This is such a good point. Take SQL. It has survived because it is well designed, and changes so little and so slowly, and it’s obvious what SQL is and isn’t meant for. Amateur programmers can port SQL queries between database systems semi-painlessly. In the right environment a query can survive with small edits for YEARS.

Who wants to break old SQL? Nobody.

That's something of an illusion. The flaws in SQL often aren't noticed because you usually pick one database vendor and stick with them. There are plenty of differences between vendors, but you don't usually have to support them at the same time, and you don't notice for simple cases.

But changing database vendors for a company can be a big deal, as bad as going from Python 2 to 3.

This is true. But, I think he means syntactically you can make jumps from one dbms to another without much of a fuss. Obviously there are non-language features but, that is not what is being discussed here.

Dealing with Unicode in SQL has long been a backwards-compatibility nightmare.

In all fairness, dealing with unicode in general has been a pain point for me.

That's mostly a result of databases (kinda) converging, SQL being a declarative language and it primarily being a wrapper around relational algebra, with a bunch of flags. And of course, it's a very direct interface to the RDBMS, and you're just migrating between systems that try very hard to have the same interface.

As a language however, it's a whole lot of nonsense. Extremely inconsistent syntax, stuffing a trinary logic into a boolean system, a standard that gets extended arbitrarily, and even the tooling ecosystem is a fair bit pathetic (the lack of formatters particularly annoy me; everyone tries to support SQLs generally, and end up missing every extended feature.. if it's not a simple select query/ddl, you're not getting a decent format output)

And it most certainly is a whole lot of fuss to migrate unless your database is tiny, or you didn't actually utilize the DB except as a dumb datastore (eg you relied solely on your ORM + indexes); there's a reason no good translator exists, and those that exist only support a very limited subset of any particular SQL variant, despite programming languages having a whole array of transpilers; it's simply not at all a simple language, and the variants only superficially look the same.

"As a language however, it's a whole lot of nonsense. Extremely inconsistent syntax, stuffing a trinary logic into a boolean system, a standard that gets extended arbitrarily, and even the tooling ecosystem is a fair bit pathetic (the lack of formatters particularly annoy me; everyone tries to support SQLs generally, and end up missing every extended feature.. if it's not a simple select query/ddl, you're not getting a decent format output)"

Would love to see your point in action. For me personally, speaking strictly writing simple scripts, they pretty much translate fairly well. Regarding formatting, are you referring to the output?

> Who wants to break old SQL? Nobody.

Every couple of months there's a new startup / dev site that says "SQL is broken/old/bad, so we reinvented it!". They all sink without trace, but there's a cohort who agrees with them.

The problem I think is they typically try to go ahead and reinvent the entire RDBMS as well.

It's not clear to me why postgres hasn't simply grown a whole array of frontends..

Agree totally. I had to climb the painful learning curve of psql because none of the front ends worked the way I wanted for some reason or another. Of course, having learned psql, I'm now scathing of anyone wanting a front end... hmm... maybe that's why ;)

I was actually thinking about the lack of SQL alternatives using the postgres engine; for example, why is datalog not simply available as an extension? Or MySQL syntax? PG implements a wide array of alternatives to PL/pgSQL (including standard programming languages eg python), but for whatever reason these SQL-alts never seem to consider being layered on top of the postgres engine.

However, the lack of GUI frontends is also really weird. I don't see why it'd be harder to support than any other DB, and afaik pg has gotten fairly popular..

true, why isn't there a MongoDb alternative that just wraps jsonb on Postgres?

This is backwards thinking.

Yes, it's expensive to upgrade from Python 2 to Python 3, but it's also expensive for the Python project to maintain 2 versions of Python indefinitely. If someone wanted other than the core Python team wants to step up and maintain Python 2, they are free to do so, it's open source. But failing that, expecting the Python team to support the older/ less functional version of the code indefinitely is unrealistic. Corporate owned languages have even shorter lifecycles for exactly this reason.

>This is backwards thinking.

And the alternative is cargo cult "newer is better".

>Yes, it's expensive to upgrade from Python 2 to Python 3, but it's also expensive for the Python project to maintain 2 versions of Python indefinitely.

On the other hand, they could progressively enhance upon a backwards compatible single 2 version. JS manages to do that just fine, as does Java...

Both Java and Javascript are giant messes.

Java is nothing like a mess, and JS is going from strength to strength.

  int max = new Max(10, 5).intValue();

Not sure what's that supposed to be. What's this "Max" class?

In any case, that's not some counter-argument, even if it points to a real wart.

It's "let me throw a random Java wart, as if it means something, and as if other languages don't have their own warts".

And yet, somehow, Python manages to be a bigger mess than than both of them combined.

JS and Java can do that because they were designed reasonably well from the start. All they tend to add are more features on top of the solid core language. Python's language was not solid (e.g., strings not unicode by default) so they needed a major overhaul.

Kind of an odd slam, given that Python (and most other widely used languages) predated Unicode.

In any case, I think the jury is still out on whether Unicode in the primary string type is a good idea.

Didn’t C# come out after Unicode (1990s vs 2000)? And they still didn’t implement it properly. Instead, they went with the horrible UCS-2 because it made interfacing with the Windows API easier (it uses wchar_t (16-bit) and UCS-2).

I wouldn't call Js a reasonably well designed language but at least it supports Unicode strings.

> JS and Java can do that because they were designed reasonably well from the start.

Javascript was knocked up over a week or so. Sure it implements concepts from Scheme and other languages but it was certainly not "designed reasonably well from the start". Otherwise we wouldn't have needed books such as Crockford's "Javascript The Good Parts" to help us understand areas of the language to avoid/misuse.

> Python's language was not solid (e.g., strings not unicode by default)

From the article:

"Python itself predates the first volume of the Unicode standard which came out in October 1991."

It's funny that you should bring up Unicode strings as an example. Java strings aren't Unicode by modern standards - their char is fixed as 16 bits, because back in the day, UCS2 was "good enough for everybody". So the moment you have, say, an emoji, stuff like length() and indexing is no longer dealing in actual codepoints. And you can slice in the middle of a surrogate pair, and end up with an instance of String that's not even valid UTF-16.

Python didn't get Unicode until later, so it had a chance to do it right - and it finally did, even on platforms like Windows where wchar_t is also 16-bit for historical reasons.

>JS manages to do that just fine

How do you define just fine? It's taken us many years to migrate EMCA versions only to have multiple incompatible runtimes.

And JS "The good parts" is like 1/10 of the full language so often it feels like a lot of pile on.

>as does Java

How are them generics?

Now do C#.

You are moving the goalposts and ignoring the fact that Python3 still didn't deliver anything for most users.

This is a hugely American point of view. For anyone who has to deal with unicode on a regular basis, the better unicode support alone is a huge improvement. That's without even looking at the advantages of Async support which offers big performance benefits for web developers—roughly 70% of Python users.

Fair criticism. But shouldn't you be even more pissed that the Unicode release was botched and we're still talking about it more than 10 years after?

When I upgrade to a new version of C# ... nothing happens.

Backwards compatibility is what made Microsoft the company it is. I think Python deserves all the crap it gets for 2 vs 3.

They have to be backwards-compatible to persuade people to pay for their proprietary products rather than switch to another vendor.

So you're saying backwards-compatibility is awesome. I agree.

Note that the question is not "Shall we support Unicode?". Clearly we should.

The question is rather whether it would have been better to gradually improve support for it in a Python2-esque way, rather than creating a discontinuity and a raft of new problems, some of which linger to this day.

Also, for many purposes, there is wide agreement that ASCII is still the way to go. Even if Americans vanished tomorrow, the majority of remaining programmers in the world would prefer to look at source code in English, which they already know, rather than a host of other languages, most of which they don't.

You are replying out of context here. The comment I was replying to was specifically "Python3 still didn't deliver anything for most users". Which is just not true.

Unicode support isn't about supporting unicode in files of source code!

They did try to do it in Python 2 and intruded Unicode type. The problem was that not only it didn't help, it is the main reason why migration to Python 3 is so painful.

There was no way to gradually improve Unicode support in Python without breaking things, because a big part of it was stuff like implicit str/unicode conversions - that were broken because they practically never use the right encoding, but that you can't remove without introducing as much breakage as Python 3 did.

You lack imagination here :) There are several solutions that could have been pursued, including introducing a completely new type and effectively duplicating the existing string library for it.

It's not just the string library that is affected. It's literally every API in the stdlib that returns a string. You'd have to fork all of those, because changing any of them to return a completely new type would be a breaking change as well.

Replying here because the other is nested too deep.

You do realize that C# was itself Microsoft's replacement for C++ right? And that when C# was released it had it's own growing pains and long roll-out in spite of having the worlds biggest corporation pushing it.

Python as a language is far older than C# so it had a lot more baggage than C# does.

> How do you define just fine? It's taken us many years to migrate EMCA versions only to have multiple incompatible runtimes.

ECMAScript versions are forward-compatible. Any valid ECMAScript 3 code is also valid 5.1, 2015, 2016, etc.

I'm not sure what migration you're talking about. If you mean using new language features before your runtime targets support them, that's kinda on you. Even so, the ecosystem has tons of robust solutions for supporting legacy interpreters. Most notably, Babel does a wonderful job of transpiling to lower language-version targets.

>How do you define just fine?

Besides the total domination of the web programming space, which is of course aided by it being the only option:

1) Used by choice even on the server and application development (where it was never the only option, and wasn't even preferable/viable before)

2) Fast pace of language development

3) A thriving package ecosystem with millions of packages

4) Adopted by all major companies

5) Three best of class runtimes (v8, JavascriptCore, Tracemonkey (?)) by 3 major vendors, with performance that smokes any dynamic language that is not LuaJit

6) Increasingly adopted as an embedded scripting language in all kind of apps

7) With a viable gateway into both a native trans-language runtime (webassembly) and a typed version of the language (typescript).

>How are them generics?

They're doing great. It's not that type erasure is that big of a deal, and Java might even get it with Valhalla eventually anyway. It's not a "backwards compatibility prevents this" issue (which is our topic here), it's a "no time devoted to add it in yet" issue.

And yet Python is eating the world: http://pypl.github.io/PYPL.html

Not that I'm implying that 'popularity' is a good measure of anything.

Javascript is the king of compromises. It's supported by every browser on the planet so of course support is massive. It's not as if a front end web developer can choose to work on the web and not use Javascript in some form or another.

Python is popular largely based on the fact that it's so approachable. It is the BASIC/ VB of modern times for whatever that is worth. It does scale up to larger projects and is frequently used for big scale stuff, but I suspect the fact that it's so ubiquitous has more to do with the fact that it's also easy to pick up and for companies to find people with Python dev skills (or train them up).

Then what are you implying?

Perhaps that listing all the excellent reasons for why some tech is 'better' than another does not translate into said tech being used.

Almost everyone hates both Java and Javascript.

> Almost everyone hates both Java and Javascript.

I just don't think that's true.

And I dislike Python, but that's at best a cool story, bro.

This is a story about language upgrades. Those languages showed how to do it right.

Javascript is a hugely popular language, as is Java.

Some hipsters hate "Java and Javascript". The world at large loves them.

At some point plain users hated Java applets and Java desktop apps, but those are not a thing much more. In the server space, very few that use it hate Java, and millions use it.

Just because they're popular doesn't necessarily mean their users like them.

Lots of people learn and use these languages because:

1 - That's what they're taught at school.

2 - That's where many if not most programming jobs are.

3 - There are a bazillion libraries they can use, compared to other languages.

4 - JS is built in to browsers.

5 - They don't know any better.

6 - Marketing.

Also, many companies want their staff to develop in these languages because of the reasons on this list plus that's what most programmers know, so it's relatively easy to find employees.

Ah yes, let's turn this into a good old "your language sucks" thread

> On the other hand, they could progressively enhance upon a backwards compatible single 2 version. JS manages to do that just fine, as does Java...

Common Lisp has a backwards compatibility which goes into decades, and implementations like SBCL had no difficulties at all to absorb Unicode.

Racket even supports different language standards and completely different languages (such as Scheme and Algol) running on the same runtime. And both SBCL and Racket are compiled languages with a high-end GC which should make such things more difficult than CPython, which is purely interpreted and has a simpler GC.

But the incompatibility between Python 2 and Python 3 is perhap s only a symptom of a larger problem. The Python developers have decided that backwards compatibility is not that important any more. This is not a problem for companies like Dropbox, or small start-ups, from which 95 % will not even exist in five years on. It is, however, a huge problem for domains like scientific computing, where most code has no maintainers and even for very important code there is no budget or staff for maintenance:

https://blog.khinsen.net/posts/2017/11/16/a-plea-for-stabili...

> But the incompatibility between Python 2 and Python 3 is perhap s only a symptom of a larger problem. The Python developers have decided that backwards compatibility is not that important any more.

Exactly: and that was a wrong decision for anybody but the developers of Python.

Everybody else prefers having something that works: "The improvements are welcome, but please allow us to to run our old programs too, thank you, and allow us to use that new feature only once we need it."

It's an obvious expectation. We would also hate a new version of a word processor which wouldn't open our old documents. Or a new version of Photoshop which wouldn't open our old pictures. Or a new version of the browser where only the newest web pages are visible.

It follows that it was absolutely technically possible to have a new version of Python in which the old programs still work. It's the failure of the developers that they haven't made it.

Compare that decision of theirs with the policy of Linus Torvalds who insists that the old user programs should never break on newer kernels.

As an example for Linus' "kernel changes should not break user programs" policy, he was famous for using harsh words to pass the message to those who tired to steer otherwise (these words I don't have to repeat, so I'll just quote his main message):

https://lkml.org/lkml/2012/12/23/75

"How long have you been a maintainer? And you still haven't learnt the first rule of kernel maintenance?

If a change results in user programs breaking, it's a bug in the kernel. We never EVER blame the user programs."

To explain that a bit, a Linux kernel maintainer is somebody who organizes and collects contributions of others, screens them, and when they are finished, passes them on to Linus to integrate them. That's a huge responsibility, and Linus uses such strong expressions only for people who have such responsibility and which do things which would harm his project. It is not the case that he talks like that to normal contributors (which should be guided by the maintainers).

And the reason why I think there is a deeper problem is that the issue is clearly not solved with the python2/python3 transition:

Why do some python developers have to maintain installations of a whole handful of python versions just to ensure that their code is working? Why all the mess with pyenv, virtualenv, and so on? If the python developers, as well as the library developers would support backwards compatibility, this would not be necessary at all.

Obligatory Reference to Rich Hickeys talk on versioning and interface specs:

https://www.youtube.com/watch?v=oyLBGkS5ICk

Discussion:

https://news.ycombinator.com/item?id=13085952

> the issue is clearly not solved with the python2/python3 transition

Exactly. The incompatibility mess continues when using different 3.x versions.

> It is, however, a huge problem for domains like scientific computing, where most code has no maintainers and even for very important code there is no budget or staff for maintenance

I think that's a tool selection problem, not just confined to the python world. If the language and libraries won't have a supported lifespan that matches with the maintenance budget of the projects using them then the wrong tool was chosen. If a project is expected to have a 10+ year lifespan of little to no maintenance then it needs to be built on languages/libraries that will have supported versions for that long.

Well, first thing is that most new code in scientific research is developed in PhD projects which last some three or maybe four years. The people who develop this do not have resources and time to maintain this code. Projects don't have a budget for that. There are projects which are very long-running (think CERN or ESO's VLT) but even there the true duration of the code usage is seldomly planned (AFAIK, ESO VLT has just started to transition from Tk/Tcl to Python).

You could say then, "well, then Python is perhaps just not a good match for those pesky scientists".

And this brings up two more points:

* A lot of important tools and libraries in the Python ecosystem was developed by scientists. Numarray/Numpy is a good example.

* If the core Python developers don't have the intention to maintain a backwards-compatible language version for more than, say, 15 years, they should perhaps clearly state on the python.org main page something like: "great, as you are a scientist, we welcome your contribution, but Python might not be suitable for tools that support long-term research".

"ESO VLT has just started to transition from Tk/Tcl to Python" - they might regret this, new Tcl releases tend to take backward compatibility much more seriously than Python :-)

Yes, I've always thought that Tcl should be promoted as an archival quality programming language.

> It is, however, a huge problem for domains like scientific computing, where most code has no maintainers and even for very important code there is no budget or staff for maintenance:

Data science is doing just fine, in fact is leading the migration: https://www.jetbrains.com/research/python-developers-survey-...

"data science" and "science" are quite different things. Science is the systematic and collaborative pursuit of knowledge in a long-term endeavour. It is based on sharing and open exchange of methods and tools. Numericas, mathematics and computational codes are just important tools to do that. As Hinsen in the blog post I cited above points out, the most part of important computational codes is written in one-off research projects which go for a few years, and the people who develop these codes normally have to move on and work for a different institution, if they manage to keep working in science. On the other hand, important codes and algorithms may be used for many many years.

"Data science" is a broad term but usually just means the application of numeric, and sometimes scientific, tools to commercial means. It is almost always done in companies. Typically, between such companies there is no open exchange of tools and methods, no exchange of knowledge, and no long-term use of generated codes. This is the reason why data science companies don't have the problems which Hinsen pointed out. But, they could become affected by a degrading suitability of Python for computational science, because their tools were initially developed by scientists.

Discussion of Hinsen's post on HN:

https://news.ycombinator.com/item?id=17058269

> And the alternative is cargo cult "newer is better".

Of course it's a "cargo cult" when someone disagrees with you.

> On the other hand, they could progressively enhance upon a backwards compatible single 2 version. JS manages to do that just fine, as does Java...

"Just fine"... that explains why so many shops are dropping support for straight Javascript and switching to TypeScript or CoffeeScript before that. And why Javascript is littered with band-aid libraries like Underscore that are needed to turn it into an effective development language.

Likewise, Java development is slowly being superseded by Kotlin. Java is a mess, there are often 3-4 ways to do simple things and many of them are just terrible for performance.

>Of course it's a "cargo cult" when someone disagrees with you.

No, it's obviously "backwards thinking", right?

>* "Just fine"... that explains why so many shops are dropping support for straight Javascript and switching to TypeScript or CoffeeScript before that.*

CoffeeScript was just adopted (and not that much in the first place) because it brought new syntax/features earlier. Now JS has been getting new syntax itself at a great pace and CoffeeScript just died off.

As for TypeScript, this is just Javascript + type annotations. Kinda like what Python is getting with 3.6 and mypy, but more useful and with actual tooling available. So not sure how "TypeScript or CoffeeScript" prove anything about JS not doing great.

>Likewise, Java development is slowly being superseded by Kotlin. Java is a mess, there are often 3-4 ways to do simple things and many of them are just terrible for performance.

Python has 10x+ worse performance, and more than 3-4 ways to do simple things (from package management to basic libs), most of which are just terrible for performance.

Compared to that, nobody has had any problem with Java performance for 15+ years...

And Kotlin is still insignificant except in the Android space where it's pushed, so there's that. Java sees an order of magnitude more usage.

No, 'cargo culting' is when you don't understand the causal link between cause and effect. Of course "newer is better" in this case, and we know exactly why: Thousands of man days have been spend improving underlying libraries, lessons have been learnt, new ideas stolen from other languages, optimization, security improvements and so on.

> If someone wanted other than the core Python team wants to step up and maintain Python 2, they are free to do so, it's open source.

Only if they name it something completely different from python or py-anything. Guido refuses to allow anyone to just step in to maintain py2.

Tauthon is a project that aims to keep compatability with py2 while adding whatever features of py3 that won't break py2 and to have a maintained py2.

https://github.com/naftaliharris/tauthon

> Yes, it's expensive to upgrade from Python 2 to Python 3, but it's also expensive for the Python project to maintain 2 versions of Python indefinitely.

No maintaining 2 versions of python is much cheaper, it's only being done in one place compared to the thousands and thousands of python 2 code bases you'd have to convert.

It also only needs bug fixes, there are plenty of people/organisations out there that would be perfectly happy for the language to be unchanging.

That's only true if you ignore all packages on PyPi.

It's extremely hard to keep compatibility with Python 2, many authors can't wait to do the support next year, many already dropped.

> That's only true if you ignore all packages on PyPi.

Presumably any packages worth maintaining will have far more dependent projects, so it's still far less overall effort.

> It's extremely hard to keep compatibility with Python 2

So don't? I don't think most of the people dragging their feet on the upgrade need or even want new features. A stable python 2 branch with bug fixes and security patches would suffice for most and be ideal for many. Over time the bug fixes should trend to zero and there probably aren't a heap of security issues in python projects anyway.

I suspect that someone, or a group of people, will step up to unofficially maintain Python 2 for the foreseeable future. It's clear that there are a lot of people using it that either can't easily migrate or are unwilling to do so for the various reasons already discussed in this thread.

I'm sure that for the right money you can find someone.

Don't forget that that person/organization would not only have to maintain python but also all the packages that you will be used.

If I encounter Python 2 bugs that matter to me, I can and will fix them myself if needed, and submit or otherwise publish the change.

I'm quite certain I'm not the only one.

Why not just migrate to 3?

genuinely curious...

Because of the amount of work involved in porting my existing Python code. Python 3 doesn't offer any advantages that matter to me, so that's a lot of effort for little gain.

OK, thanks. Good luck with the bug fixing ;)

I'm not really worried about bugs, because the code I have works well, and I doubt I'm going to do any serious new Python 2 development.

I meant the bugs in your dependencies that you'll have to backport fixes to. But hey, if you're writing bug-free code that'll be easy ;)

I understood what you meant. What I'm saying is that my existing code works fine, so whatever bugs are in the dependencies are ones that don't affect me. Should I make a code change that surfaces one, then I have the means to deal with it -- but that is almost certainly going to be a rare event, as my Python projects are stable and aren't going to see much change.

I wasn't commenting on how buggy my own code is. Which version of a language I'm using doesn't really affect that variable.

I get that. But the interesting thing about dependencies is how they surface vulnerabilities that can hurt code that works perfectly well. Your current code probably doesn't have many bugs, but includes an unknown number of vulnerabilities from your dependencies. The bad people probably won't bother examining your code for vulnerabilities, but they will be informed of vulnerabilities in popular libs, and then looking for projects that use those versions of those libs is a lot easier than scanning all those projects individually. So you end up having to backport a bunch of fixes to other people's code because that code was popular and came under intense scrutiny.

But I guess you know this, and are OK with the compromises involved. I'll stop here ;)

> but they will be informed of vulnerabilities in popular libs, and then looking for projects that use those versions of those libs is a lot easier than scanning all those projects individually.

This is true, and if we were talking about code that is exposed to the world at large, then my stance might be different. However, the projects that I've used Python for are not exposed in that way.

Note that I'm talking about personal projects, not work-related ones. At work, I use whatever is required.

Reply on news.ycombinator.com