Hacker News Reader

Comment by x1798DE | Hacker News Reader

I don't understand why it was less tedious to write a source-transforming module than to just make the static code changes once and be explicit about it.`b''` is supported in Python 2.7, so it's just redundant. I think people will be confused if they look at a nominally Python 3 codebase and find that string literals are being encoded as bytes.

It's certainly a clever hack, I just think you're more likely to introduce "subtle bugs" when you are doing something decidedly non-standard implicitly.

View on HN

Having been involved in several large scale (10s KLOC) source ports from Python 2 to 2/3 compat, I can say that IME it is trivial to make a mistake when handling these kind of changes. The complexity and regression risk is entirely dependent on the complexity of the codebase and I wouldn't be surprised if that factored fairly heavily into the Mercurial package teams' thought processes. The case for spending valuable people time on the harder problems in their codebase seems like a valid choice if the scale is large. TBH I wish I'd known about this a year ago...

It's also worth noting that the end goal may also not align with valid intermediate goals. I often found myself fretting about where to pivot to using a byte literal prefix, where to accept an unprefixed literal (to get pre-3.5 % formatting since the codebase used that in some places) and where I explicitly needed a unicode literal prefix (and dealing with the headache of what to do if we cared about 3.2 - the brief wasn't specific enough at first).

Just $0.02, I think everyone's experience, expectations and desire for purity will be different for porting efforts. It's worth noting that my porting work was supported by a large number of anally retentive unit and regression tests which made the effort much more frustrating but a heck of a lot more obvious when we messed it up!

When you change the source code, you have to take care to test on both Python 2 and 3. With this solution, the transformation (I think) only happens on Python 3, so there is no way to introduce bugs in Python 2 at all. Which is an important thing if your code base is big enough.

While explicit is better than implicit the practicality still beats purity.

EDIT: yup, only happens on Py3: https://www.mercurial-scm.org/repo/hg/rev/1c22400db72d#l2.10

But that makes things worse, because you have two different versions, one of which you can't see directly. b'' is the same as '' in Python 2 in the same way u'' is the same as '' in Python 3.

Since this transformation only has any effect on Python 3, it doesn't matter if you apply it in both 2 and 3.

By "this transformation" you mean manually going through the whole hg code base and inserting "b" characters before most string literals.

Even if you are 100% sure that semantically `b""` and `""` in 2.7 (let's forget about <2.7 for a moment) don't differ, you cannot be sure that the transformation itself - i.e., editing the source files - won't break something. You have two ways of doing this: either manually, where you risk human error, or by writing a tool, where you risk missing certain cases. In both cases, your perfectly good Py2 code base breaks for no good reason. Moreover, even if you create a tool for applying the transformation (actually, it looks like even this loader could be used for this) to the source, once you run it and commit (imagine the commit diff and its review process!), you can't run it again if you discover your tool missed something (so iterating and improving the tool becomes harder).

With the solution described in the article, you have 100% guarantee that nothing breaks on Py2 and simultaneously you can work on improving the tool. Once they're satisfied with the tool performance and the Py3 porting is more advanced, they can use it to convert the source files. IMHO that's a really sane strategy here.

Reply on news.ycombinator.com