Comment by mangecoeur | Hacker News Reader

Comment by mangecoeur | Hacker News Reader

I get that Armin runs into pain points with Py3, but on the other hand I get annoyed with the heavily English centric criticims - its easy to think py2 was better when you're only ever dealing with ASCII text anyway.

Fact is, most of the world doesn't speak english and needs accents, symbols, or completely different alphabets or characters to represent their language. If POSIX has a problem with that then yes, it is wrong.

Even simple things like french or german accents can make the Py2 csv module explode, while Py3 works like a dream. And anyone who thinks they can just replace accented characters with ASCII equivalents needs to take some language lessons - the result is as borked and nonsensical as if, in some parallel univese, I had to replace every "e" with an "a" in order to load simple english text.

Armin is Austrian. Whatever else you think of his critique, it's probably not English-centric and he's had to deal with accents in his native language.

My libraries are all supporting unicode on Python 2. And in fact, they do it better than on Python 3. File any unicode bugs you might encounter on Python 2 against me please.

This is true, and in fairness I've never had any problems with unicode using any of your libraries, probably because you take a lot of care in explicitly dealing with encoding.

But that's not always the case with other libraries like the csv module. The core unicode support in py3 means that a lot of librares which are not written with explicit unicode in mind Just Work with it in py3, and its a huge time saver.

This is not what the blog post is saying. It is saying that Python 3's attitude of forcing Unicode is making life difficult, whereas in Python 2 it is easier to decode to Unicode where needed, and be able to accept non-Unicode data in other cases. That Unicode needs to be supported was never under discussion.

I've used the Py2 csv module with csv files containing accented characters and had no issues. Could you post an example?

This works fine, as does the corresponding writer:

    reader = csv.reader(open(FILENAME, 'rb'))
    for row in reader:
        print row

If you want a unicode string it's as simple as:

    value = row[0].decode('utf8')

Then before writing:

    row[0] = value.encode('utf8')

Reply on news.ycombinator.com