Hacker News Reader

Comment by gtufano | Hacker News Reader

I can tell that proper Unicode support is a reason to discounting a language/library if you're not a native English speaker and/or writing programs that have to be localized.

View on HN

Python was created by a non-native English speaker.

I know that. I'm Italian, and Western European Languages can be expressed on a 1-byte character type. I also understand that the performance gain of 1 byte char was significant. But now is 2014 and not to have to wrestle with encodings definitely worth the performance and (somewhat) complexity hit.

As a unrelated (to Python) example of the encoding nightmare: have you ever tried to import text from CSV files generated on different OS/locales in Excel that don't support Unicode-encoded CSV files? It can be made, sure... but, man, if it can be avoided... ;)

I did this recently. I can't remember the details but it was absolutely insane.

Well, he's Dutch, his language uses an alphabet meaning that even with all the accents it's still a one byte language.

Having to work with 2 bytes languages (CJK typically) without proper Unicode support is much more of a hassle.

> Having to work with 2 bytes languages (CJK typically) without proper Unicode support is much more of a hassle.

Proper unicode support is not even close to being a priority when it comes to dealing with Japanese text. No one would store important Japanese text in unicode. http://en.wikipedia.org/wiki/Han_unification#Unihan_.22abstr...

One size doesn't fit all.

This makes me wonder about Ruby, whose creator is Japanese and Japanese is certainly not a language whose characters can be expressed in a single byte. Is Ruby better than Python for complex encodings? If so has it always been this way, or did Matz just deal with it in the early days?

Yes, Ruby has long had decent (i.e. comparable to Python 3) encoding support. But in the early days not much was made of this, perhaps because Rails was the driving force behind Ruby adoption in the west and Rails' encoding support was much worse than what Ruby could do.

Before Unicode existed

And Python3 was also endorsed by the same person, so?

Reply on news.ycombinator.com