Hacker News Reader

Comment by csense | Hacker News Reader

> the thing should be called Zöpfli (note the umlaut)

No, it shouldn't.

A few years ago I downloaded a tarball of Java code. Every file had one of those "ö" characters, because the author spelled his name that way, and his name was in a comment at the top of every file as part of his license notice.

The project would not compile.

I tried compiling with Eclipse. I tried the included Ant script. I tried compiling directly by typing a javac invocation into the command line. I tried every permutation of the locale environment variables that Google knew about. My environment was very conservative, an official Sun Java toolchain from apt on a recent Ubuntu.

Finally, I wrote a script to delete that line from every file. It compiled just fine after that!

Putting anything but ASCII in source code will give everyone who ever tries to build the code endless nightmares of toolchain breakage.

View on HN

So what you are saying is that Java was broken. Not that using characters containing an umlaut is a terrible idea.

The world is no longer ASCII, unicode is here to stay. I don't see that disappearing anytime soon, get used to it.

Speaking of which, code I write in Python that deals with ØMQ has the Ø in it ... i've found some editors that don't deal with it correctly, but that is the fault of the editor, not the file.

> The world is no longer ASCII, unicode is here to stay.

I wouldn't have a problem with it, if Everything Just Worked. If everything international was UTF-8 and Just Worked, I'd be cool with that. But:

1. > some editors that don't deal with it correctly

2. You need fonts installed, otherwise you get boxes everywhere.

3. You need to specify locale somewhere. This is currently sometimes manual: Applications can't automatically agree on locale, because not everybody uses UTF-8, and not every format that can be in multiple locales puts it somewhere.

4. Linux insists on "generating locales." Now that's a mess.

Now people who want to show how edgy they are by insisting on using the empty set symbol are wasting my time, because when I install $COOLAPP that depends on $AWESOMELIB which requires $YOURPYTHONZEROMQLIB, then I discover after hours of Googling cryptic error messages and browsing multiple forums for these different projects (plus the OS(es) and programming language(s) involved), that your Python was compiled with wide-character support, and I have to bootstrap a wide-character-compatible gcc in order to build wide-character-compatible Python that can run $COOLAPP, but I have to also create more locales and change the LC_ALL environment variable, which (when I finally do it) will of course cause another obscure error message, and it turns out that $OTHERCOMPONENT really doesn't want to use wide characters because it only knows UTF-8, and there's really no way to have multiple processes in the same address space because different character widths can only run in different chroots because they're like amd64 and i386 before Ubuntu grew multilib support...

I haven't had this exact breakage, but the point is that this is the sort of rabbit hole you can find yourself falling down, all because you wanted two dots over your 'o', or a slash through it, just to show how edgy and hip and modern you are.

Fortunately you have contact information in your profile, so when your code does this and I end up wasting 10 hours of my life with this garbage, I know where to send the invoice.

This is incorrect. Java supports Unicode in source files just fine, as long as the file encoding is correct.

The problem is that certain IDEs (such as Eclipse) default to ISO-encoding on Windows. This works fine as long as you're only using Windows, but it will fail on Linux.

At my previous job certain developers preferred Windows and ended up commiting ISO-encoded source files which our build server (Ubuntu Server) failed to compile for this exact reason. A quick recode(1) usually resolves it.

Hey, just because java has a stupid flaw is no reason to infect the rest of the world.

> Putting anything but ASCII in source code will give everyone who ever tries to build the code endless nightmares of toolchain breakage.

Only if you use garbage tools.

It's a tool problem.

But figuring out exactly what part of the tool is buggy is hard. If I'd have figured out what the problem was (probably spending hours), and reported it, the javac people would've said, "javac is working just fine, you specified the wrong locale." The Eclipse people would've said, "Eclipse is working just fine, it uses the system locale as a default when you don't specify." The Ubuntu people would have said, "Our locale setup is fine, it's your fault for not selecting ISO-1234-5678 instead of or in addition to UTF back when you installed the system. Oh, and of course you also have to set your environment variables correctly..." The author would have said, "I'm using a normal system and it compiles fine for me. If I change encoding so it compiles for you, it breaks on my system!"

The upshot is, the tool support is garbage, because you have to jump through a bunch of hoops to get code to compile. I recall the error messages not being helpful. (Would it be so hard for the compiler to say "Warning: Unrecognized character o.., it looks like you're using ISY-2358 encoding even though you didn't specify. To make this warning go away, enable the ISQ-1337 locale, regenerate your locales, and set the LC_ALL environment variable.") Being able to compile a given piece of source code into object code should not break if you change obscure environment settings between machines.

Spending hours figuring out what's breaking, only to find out that things aren't breaking and I have to change a bunch of settings on every development machine I use (with different values for every program I compile, of course), is a lot more effort than just limiting your source files to character codes less than 128 (or 126 or whatever the top ASCII character is).

Many languages explicitly allow you to use Unicode even in variable names. Your issue was a limitation of the language, though I can see the issue with using the name in a compression algorithm that has the potential to be used in many languages that might not deal with the non-ASCII characters all that well.

Reply on news.ycombinator.com