> six is non-optional for writing code for Python 2 and 3
I maintain a Python 2 & 3 compatible project that has no external dependencies.
I maintain a Python 2 & 3 compatible project that has no external dependencies.
In some cases of common OS-induced pain, I'd say "do whatever 3 does" in 2, since that'll make migration easier in the long run. (But I understand that can be hard, and I think my responses to your examples below even demonstrate that to be hard.)
To your specific pain points:
> Unicode stdio?
Mostly, `io` should handle this in both 3/2. You might need to help it get the right encoding in 2.
> Unicode file paths
This is going to be a mess in any language, because file paths really aren't text. On nix, they're byte strings that don't have nuls in them. Hopefully* they're encoded according to LANG, and hopefully LANG is a UTF-8 variant, but it isn't required, and it isn't required that two users on the same system use compatible LANGs, so you get a Tower of Babel. I really wish OSs would just start enforcing a. Unicode filenames, and b. no newlines in filenames; those two alone would make life so much easier.
Hopefully you've seen os.fsencode / fsdecode, but alas those aren't in 2, so I'm not sure they really help you. Often one is not really munging paths that much, and can just pass through whatever value/type you get, but it does happen, of course. (E.g., adding or removing extensions)
> Unicode sys.argv
This is also a pain point, since again, the underlying type in nix is a byte string without nuls. I'd hope it decodes w/ the LANG encoding, but since the user could easily tab-complete a filename, fsencode/decode might be more appropriate. I think I'd say "do whatever 3 does".
1 Jan 2020 is nearly here. Forget about 2 / assume UTF-8 in 2 and don't support anything else?
> I feel like Python 3 completely wrecked strings instead of making them better.*
A clear separation of text and binary is needed in the long run, and makes other operations much clearer and saner. The pain you're feeling is introduced from the OS not having the same clarity.
> Mostly, `io` should handle this in both 3/2. You might need to help it get the right encoding in 2.
Does io handle stdio? I was referring to standard input and standard error/output. How do you read/write Unicode in a cross-2/3 way from/to standard input/output/error without adding your own translation layer?
>> Unicode file paths
> This is going to be a mess in any language, because file paths really aren't text.
Sorry I need to be more clear. That general mess is not the aspect of it I was referring to. I'm specifically referring to a 2/3 compatibility mess.
I meant that, for example, to have any semblance of Unicode handling, in Python 2 you do os.listdir(u"."), whereas in Python 3 you do os.listdir(b"."). I know how to handle it in both Python 2 and Python 3 in a way that's Good Enough (TM), but how do I even get that with cross-compatible code? I'd need to write a translation layer of sorts for every single I/O function I might use.
>> Unicode sys.argv
> I think I'd say "do whatever 3 does".
Hmm okay thanks, I'll need to try to see what the implications are again. I think the problems I recalled from this may have been just been a result of the other issues, not sure.
I can't "forget about 2" though, it's still on Ubuntu LTS systems and there are still packages in 2 that haven't been ported to 3.
>> I feel like Python 3 completely wrecked strings instead of making them better.
> A clear separation of text and binary is needed in the long run, and makes other operations much clearer and saner. The pain you're feeling is introduced from the OS not having the same clarity.
Again, I think this is "cleaner" in theory, not in practice. What happened to the string_escape/unicode_escape nonsense I pointed out with the new system? Any rebuttal to that one? ;-)
No. No on just so many levels.
It finally fixed it by introducing two (or three if you count bytearray) types for completely different semantics of data.
Python 2 was just a mess whenever you left ascii.
But looking solely at python3 , everything byte and unicode related just got so much easier and better testable and with better error messages at better points in your code than in python 2.
So claiming python 3 broke stuff just not what the case is.
Python 2 had a deeply flawed, boundryless view of unicode vs bytes and python 3 fixed that.
What is your reason for using this obscure encoding anyway?
Yes and my entire point is that this makes zero sense when we're talking about escaping.
> What is your reason for using this obscure encoding anyway?
Seriously?
The encoding `unicode_escape` is not about escaping unicode characters. It's about python source code. It's defined as:
> Encoding suitable as the contents of a Unicode literal in ASCII-encoded Python source code, except that quotes are not escaped.
It makes absolutely no sense to have escaped unicode characters as actual unicode string. If you really need that, a version that also works in python 2 would be:
u'hellö'.encode('unicode_escape').decode('ascii')Absolutely no sense? So is a basic Python expression evaluator like this complete nonsense to you? #!/usr/bin/env python3
try: # Python 2
from Tkinter import Tk, Entry, END
import tkMessageBox as messagebox
except ImportError: # Python 3
from tkinter import Tk, Entry, END, messagebox
import ast
def my_eval(s):
# assume I've implemented this functionality manually...
return ast.literal_eval(s)
root = Tk()
e = Entry(root)
e.insert(END, '"Hell\\xc3\\xb6"') # Assume the user typed has typed in a Python expression, not me
e.pack()
root.bind('<Return>', lambda evt: messagebox.showinfo("Result", repr(my_eval(e.get()))))
root.mainloop()
Note that that wasn't even my choice! That was the choice of the built-in Python GUI toolkit... distributed by the same folks who decided this string/bytes overhaul was a brilliant idea...
After A TON OF PAIN I decided to put the time in to watch the talk "Pragmatic Unicode, or, How do I stop the pain?"[0] by Ned Batchelder and it all just clicked. Now, I make unicode sandwiches like a boss.
Does writing python 2 code stop being 'worth it' at some point? If so, where is that point? It sort of sounds like you're already there. There's still people using P2 at least in part because of all the hoops people are jumping through to keep supporting it, no?
I'm not a professional python dev, so it doesn't impact me as much as some of my colleagues, but 'backwards compatible' issues crop up in other platforms/langauges as well. Wordpress might be the biggest example in PHP. They've kept a minimum target language which is far behing 'current' or even 'currently supported', and it's been a catch 22. Hosts keep supporting PHP 5.4, for example, far later than they 'should' have, because people kept writing new stuff targeting PHP 5.4. WP 5 was, alas, a missed opportunity to target PHP7 as a minimum. :/
How do you know I'm there? If nothing else I still have my own previous Python 2 code that I've spent time on and that's useful to me! Why would I just throw them all away or waste massive amounts of time rewriting them into Python 3? Is my goal supposed to be to please the masses here?
Honestly that's sufficient reason already. But if you want a more "standard"/politically-correct response: Python 2 is still being supported, Ubuntu 14.04 LTS literally only just reached EOL last month and can be found in the wild, Python 3 support is lagging in a lot of places (e.g. PyPy is still on 3.5, although I'm just using it as an example; I don't use it much), and I still come across Python 2-based tools and packages once in a while.
So I guess the answer to your question is it "stops being worth it" when I stop coming across situations where I'd regret abandoning Python 2.
> There's still people using P2 at least in part because of all the hoops people are jumping through to keep supporting it, no?
"Because" is an odd way to put it... it's true that if they didn't support P2 then people would use P2 less, but people use it because it benefits them, not "because it's supported". I mean, you're also alive "at least in part because" of all the hoops people jump through to grow and bring food within your reach, but I doubt your conclusion is that this should stop being the case...
> They've kept a minimum target language which is far behind 'current' or even 'currently supported', and it's been a catch 22.
What I don't understand is why are people supposed to keep abandoning good software just because someone made something shiny and declared it "unsupported"? I hate this "you have to like my updates... or I will force you" attitude that every organization seems to have nowadays. People are trying to solve their own problems, not please the leader they're following.
> Hosts keep supporting PHP 5.4, for example, far later than they 'should' have, because people kept writing new stuff targeting PHP 5.4.
I mean, this isn't even the same situation? I'm not targeting Python 2 or introducing a dependency on it generally. I try hard to keep my code both Python 3 and Python 2 compatible. So the decision as to whether to move on or not is left to the client (which is often myself) and there's no obstacle either way.
Part of the argument for using much of the stuff out there is because it's "supported" (commercial, community, etc). Without some degree of support, things dwindle/die. The 'support' for language stuff may simply be security patches. If/when those stop being provided, you're starting to do a disservice to people to continue to push that language, even if the core functionality you need is still working and perhaps even unaffected.
> I'm not targeting Python 2 ... I try hard to keep my code both Python 3 and Python 2 compatible.
Given that there are breaking changes between the two, I can't read this any other way that that you're 'targeting' Python 2. If there's a way to do something in 2, and it's not working that way any more in 3, and you're making code so that it runs in both (weird syntax, translation layers, etc) then... you're targeting 3. The same way that if you use 3 syntax, you're 'targeting' 3. That's just how it is.
> but people use it because it benefits them, not "because it's supported".
The benefit is that they don't have to go through whatever hoops it takes to upgrade to have the latest versions. In many cases the later version may not add much directly, but 'community support' and 'security' are two ... emergent properties of a critical mass adopting new versions and dropping support for old versions.
> How do you know I'm there?
Wild-ass guess from the gist of all your earlier posts in this thread?
> But if you want a more "standard"/politically-correct response: Python 2 is still being supported,
I'd say that's partially 'under duress'. My reading of the situation years ago was that the Python community - at least the leadership - wanted to EOL P2 on Jan 1, 2015. There was a gigantic stink/pushback, and support was extended for another 5 years. I will grant that the v3 switchover was pretty bad - I'm not a pro user, but had some colleagues that dealt with a lot, and certainly, 8+ years ago, it was pretty hard to just 'switch' running systems. Even developing new stuff from scratch - there were a lot of 'only works in 2' libraries that were entrenched standards without clear update paths. However, rather than doubling down and making 3 a more attractive proposition (work on upgrade path, faster language, better docs, whatever), the agreement was to support P2 for another 5 years. That's an eternity in the tech world. You could argue it shouldn't be, but it is. Yet, from python colleagues - those who've stayed - the upgrade story isn't significantly better. I'm taking their word for it, but based on your posts, it doesn't seem to be either.
fwiw, six can easily be vendored into a project to avoid the technical external dependency. that is how we manage it for kafka-python.
there are a few modules that are simply at different locations but have the same API
if PY3:
from http import client as httplib
else:
import httplib
PY3 = sys.version_info >= (3, 0)
PY2 = sys.version_info < (3, 0)
PY26 = sys.version_info >= (2, 6) and sys.version_info < (2, 7)
isinstance(<maybe_string>, basestring if PY2 else str)
# Python 2.6 doesn't properly UTF-8 encode syslog messages, so it needs
# to be performed in a custom formatter.
formatter_class = UnicodeLoggingFormatter if PY26 else logging.FormatterAs the maintainer of WebOb/Pyramid/Waitress we have a compat module that contains all of the changes/renames/functions to help with the Python2/3 compatibility and all tests run across both platforms.
Are the functions borrowed heavily from six? Yes, but we don't need to vendor all of six.
To be fair, one of our three dependencies does in fact depend on six, but we don't use six anywhere in our code. This dependency of ours is used only in one specific area of the codebase, and in fact I (the person who added the dependency) wish that we didn't have to use it at all.