Hacker News Reader

Comment by otakucode | Hacker News Reader

Yeah, for some reason Google is adamantly opposed to adopting Python 3. I have no idea why. It's clearly much better, more well designed, etc. I mean come on... print as a statement rather than a function call? That alone should be enough to make the decision I'd think.

View on HN

Dropbox, which is a much smaller company and codebase than Google, is (as far as I've heard) still on Python 2, despite having hired Guido van Rossum away from Google to work on static type annotations for Python to fix all the str/bytes stuff and related incompatibilities in their codebase so that they can move. Certainly this was the status in mid-2016, about four years after they hired van Rossum: https://us.pycon.org/2016/schedule/presentation/2266/

The difficult engineering problems in the world aren't greenfield engineering.

Python 3 has some architectural decisions that make working with data difficult; mostly ///my opinion/// is that it completely botched Unicode handling.

The idea of data being encoded should be an extremely high level idea, with foundations that are supported by the internal libraries and maybe even compiler.

I feel that golang has a much more data-structure friendly view of Unicode.

    * The ancient paradigm of bytes remains
    * Strings are immutable sequences of bytes; but are trivial to cast back and forth.
    * 'runes' (distinct Unicode code points) also exist

However to actually validate or do anything fancy to a Unicode sequence a programmer would need libraries from https://godoc.org/golang.org/x/text which shows how truly complex /proper/ handling of Unicode data is; instead of just the simple copy / append / search (is this inside of / a match to it) operations that make up 99% of my handling of Unicode data.

Have you worked in Python 2 unicode?? Over half of the work and debugging the search engine service management team I was on was fixing unicode document injesting.

But I agree, making unicode part of the language is very dumb. I just wish they would give up as it has never been good.

What about Python3's unicode handling is botched?

It optimized for O(1) indexing to Unicode scalar values instead of optimizing for more useful things. Also, even though the API semantics are UTF-32, the storage may be USC-2 or Latin-1, so there's a surprise performance effect when a string gets it first non-Latin-1 character or its first non-BMP character. So much for O(1) behavior.

The best in-memory representation of Unicode is UTF-8. No alignment issues, ingest "decode" is mere validation and output "encode" step is a no-op. Space requirements are favorable to common computer language syntaxes. Byte-wise lexical sort matches code point-wise lexical sort.

However, since "narrow builds" of Python 2 already used UTF-16 which allowed for things like bridging strings with Cocoa, Python would have done well to stick to UTF-16 and get rid of the "wide builds".

Not having both "narrow" (UTF-16, Python 2 on Mac) and "wide" (UTF-32, Python 2 on Debian) is an improvement, though.

Python strings are immutable. The only time a string can get its first non-Latin-1 character is at construction time, which is necessarily O(n).

Python 2 was the norm when they adopted Python internally, but I think it's inaccurate to say that Google is "adamantly opposed" to Python. They've done a good job in making their most popular open-source projects 3.x compatible, such as Tensorflow: https://github.com/tensorflow/tensorflow/issues/1

I don't think it has to be anything more than inertia. They're a massive company that has millions of lines code running in production for a whole chunk of languages. Making a change from python2 to python3 probably seems like more worth than it's worth. It is also a fairly googley problem to think that the way Google does things is automatically superior in all cases. So, since googlers use py2, why shouldn't it be good enough for everyone else?

> print as a statement rather than a function call? That alone should be enough to make the decision I'd think.

I seriously cannot tell if this is a joke or not.

Google has billions of lines of code, a non-trivial amount of which is Python. Moving to Python 3 would require an all-hands-on-deck massive effort.

You can begin a transition without moving everything at once, or even ever. Start with a policy that all significant new efforts should strongly consider Python 3 (including porting any dependencies). Keep a "scoreboard" for un-ported dependencies and add a point every time it's decided to keep a new project in Python 2 because porting a certain dependency would be too much work to identify the bottleneck. Invest in porting those. You should see the proportion of Python 3 to 2 go up. Eventually, you're left with a relatively small set of legacy stuff in Python 2, and that's probably fine to keep like that.

This works badly with monorepo and uniform build systems. Same reason why Google is still on cpp11.

That said, you can (and Google has) invest in things that make porting easier. Enforce future imports, use six, etc.

Google is on cpp11?? Impressive. Last time I read their guidelines, it read like a more conservative version of Bjarne's book from '98.

print as a statement vs. function doesn't carry that much weight when you have tens of millions of lines of currently working code to migrate.

I think the problem is that Python 3 is a different enough language that it's hard to consider it an upgrade, adapting existing code to it really is a port (although six and __future__ help).

And if you're going to port, it's not a question 2 vs 3, it's a question of 3 versus many other languages, chiefly Go.

I don't think this is true, as a python user at Google. There's momentum that has to be shifted, and that can't happen overnight. But it is absolutely a work in progress.

The writing is on the wall for py2 at Google, and in some ways it has been for years.

Reply on news.ycombinator.com