Comment by rogerbinns | Hacker News Reader

Comment by rogerbinns | Hacker News Reader

Parent Full thread

Are you expecting the same running interpreter to have both python 2 and 3 modules at the same time, with calls between the modules working? You can easily keep things separate - python 2 executable is named python while python 3 is python3. https://www.python.org/dev/peps/pep-0397/

Sadly concurrent running in the same interpreter won't work because Python 3 has different types for strings (unicode only) and a bytes type, while Python 2 has a str type (pretty much like bytes but can also be treated as a string) and a separate unicode type. Python 2 had code that tried to do the right thing automagically with str, such as automatic promotion to unicode where it looks like you really meant a string and that is what is needed. Automatic promotion can fail, be unexpected, is hard to test and numerous other issues. That is why Python 3 was needed and why it needed to be incompatible. To have Python 2 and 3 code interoperate in the same interpreter would require another layer of automatic promotion/demotion and heuristics when data goes between the two.

C++ was explicitly defined to be a superset of C so it doesn't have these kind of issues. The analogous situation would be if C didn't distinguish between long and pointer types, and C++ did. You'd need rules for how to convert between the two worlds, trying to combine or disambiguate the types depending on which way types are going. It would be a mess.

But imagine what would happen if they had created a compatibility layer. Yes, it would be very hacky, and require a lot of seemingly gratuitous work - tons of standard library functions would have to check the compatibility version of the caller and change behavior accordingly. And the goal of cleaning up the codebase by removing things like old-style classes wouldn't be achievable. But what if that work had been done, and the mess accepted?

Then Python 3 code could use every unmodified Python 2 library (albeit potentially requiring the former to do some messy conversions between str and bytes when making calls), including the long tail of legacy libraries that will never be upgraded. Meanwhile, once the issues in the first Python 3 release or two were straightened out, there would be very little reason to use the Python 2 interpreter, since the Python 3 one would be a drop-in replacement; so migration to the latter would be relatively swift, and it wouldn't be too much of a burden to ask someone to upgrade as a prerequisite to using your code. So there would be very little reason to write new Python code in the Python 2 dialect (unless you really really hate making print a function :), and existing code could be migrated one file at a time. Including popular libraries that wanted to maintain backwards compatibility with Python 2 programs - they would have to add compatibility checks similar to the standard library, but could otherwise go full Python 3, rather than having to use the awkward intersection of language semantics and compatibility libraries currently required.

With the transition so much smoother overall, native Python 3 code would probably take over relatively quickly. (Maybe it would have even been feasible to remove the Python 2 compatibility layer eventually, though probably not.) Whereas in reality, today, over 7 years later, Python 2 is still more popular than Python 3 [1].

[1] https://lwn.net/Articles/640181/

Yes, that is what I expect. The various flavors of C and C++ get along just fine that way.

C++ is not a superset of C: http://stackoverflow.com/a/1201840/270610

C and C++ have a similar issue with strings. C code uses NUL-terminated char arrays, while C++ generally uses std::string. If you want to interface the two, you need glue code (written in C++).

Surely Python 3 can work the same way: to share strings between a Python 2 and Python 3 module, you need to convert them using glue code written in Python 3. Sure it's messy, but writing polyglot code is messier.

Except that you can easily go from std::string to cstring and back in the same program. And many of the standard library functions work fine on c strings (probably over half of <algorithm>), etc. IMO theres no compatibility issue between std::string and c strings.

I wonder if there's even a reasonable way to do this in Python. In C, strings are data that you're explicitly passing around, but in Python, you are constantly calling __getattribute__ under the hood just to execute your code.

I don't know that it would be impossible to make that all work, but I wouldn't assume that it is.

You don't need to actually convert anything: Python 2's str can be identified with Python 3's bytes, and unicode with str. You 'just' need to have their __getattribute__ methods, along with a ton of standard library functions, check the Python version of the caller and change behavior accordingly. Ugly, but I think it would mostly 'just work' for user code.

Actually C++ is not a superset of C. And it does have these kinds of issues. C++ continued to diverge with C99/C11. https://en.wikipedia.org/wiki/Compatibility_of_C_and_C%2B%2B

Function pointers are one scary area. I don't know if C++11/C++14 decided to fix this yet. Most people relied on undefined behavior to make this work.

Since we are in pedant's corner, note that I very carefully used past tense. Yes it originally was a design goal.

Source: The Design and Evolution of C++, by Bjarne Stroustrup. (I don't know which printing I have.) Page 120, Section 4.5 "No gratuitous incompatibilities with C". It explains my original claim, and also why it couldn't be 100% compatible due to some other C++ desirables (eg better type safety). Partial quote:

"C++ doesn't aim at 100% compatibility with C because that would have compromised the aims of type safety and support for design. However, where these aims are not interfered with incompatibilities are avoided - even at the cost of inelegance. In most cases, C incompatibilities have been accepted only when a C rule left a gaping hole in the type system."

> python 2 executable is named python while python 3 is python3

Nope, not quite. Python 2 is named python2 and python 3 is named python3, on every system I am aware of. 'python' is ambiguous and can be linked to either [1]. I know at least on Arch Linux, python links to python3, and I have run into a handful of scripts that break by assuming python is always linked to python2.

[1] https://www.python.org/dev/peps/pep-0394/

Reply on news.ycombinator.com