Comment by Mikhail_Edoshin | Hacker News Reader

Comment by Mikhail_Edoshin | Hacker News Reader

Parent Full thread

Mikhail_Edoshin

An example: there is a text "shaping" library that takes a font, an input string and produces a sequence of glyphs to typeset that string. Modern fonts and certain scripts are very complex and this task is not trivial. Now, this particular library takes an UTF-8 string. Which means it has an UTF-8 decoder inside.

But a text shaping library does not need an UTF-8 decoder. The product it is used in will certainly have one or, if it works in UTF-16 or, as Python, uses 3-way encoding, may not even need it and thus will have to add an UTF-8 encoding step only to communicate with that library. A simpler design would be to remove that UTF-8 decoder and make the library to accept Unicode characters as integers. If we need UTF-8, it is trivial to decode a string and feed the resulting Unicode into the shaper; if we don't, it is equally trivial to use the library with any other encoding.

(I guess I ended up with a slightly different example than I intended.) Anyway, removing an UTF-8 decoder here would result in a simpler and more universal design, although - this is an unexpected development - it may superficially look more complex to many people who have the "standard" UTF-8 string and just need to get the job done.

If this makes the library harder to use because most people will have UTF-8 strings, I’m not sure that’s a win.

In other words, circling back to Brad Cox's Software ICs, we're all using devboards and Arduinos instead, because those look simple to newbies and save a little glue work here and there.

In hardware world, it's fine to use devboards and Arduinos to prototype things, but then you're supposed to stop being a newbie, stop using breadboards, and actually design circuits using relevant ICs directly, with minimal amount of glue in between. Unfortunately, in software, manufacturing costs are too cheap to meter, so we're fine with using bench-top prototypes in production, because we're not the ones paying the costs for the waste anyway, our users are.

(Our users, and hardware developers too, as they get the blame for "low battery life" of products running garbage software.)

> Unfortunately, in software, manufacturing costs are too cheap to meter, so we're fine with using bench-top prototypes in production, because we're not the ones paying the costs for the waste anyway, our users are.

I’m not sure what you’re trying to say - UTF-8 is the standard text encoding by a mile. It’s not a prototype.

UTF-8 here is like having a devboard with an USB controller chip, complete with power circuitry and USB port. It could all be high-quality components, it's super useful to have it on the board for prototyping, but in the actual product, you aren't going to ship three devboards wired by SPI, but each carrying some combination of USB, Ethernet, Wi-Fi, Bluetooth controllers and other stuff, all of it disabled/unused, all because you need three ICs and found it easier to order devboards. You're just going to use the ICs you care about, supply the USB controller and port and necessary wiring yourself, and otherwise use minimum amount of extra components necessary.

So, in context of Mikhail_Edoshin's example, I'm saying that this "text shaping library" they mention is basically a devboard - full of components not necessary for its core functions. Most software libraries are like that, so applications using them are basically like a device built from wiring up a bunch of devboards.

The reason this is so is that there is no way to say "the library accepts UTF-32, for other encodings use the standard decoder" because there is no such a decoder. "For want of a nail". So it circles back to the idea of easily composable software which is not yet there. Everybody bring their own nails and there is no way to move nails between projects.

What do you mean that there is no standard decoder? For what, UTF-8?

I agree on the composability. Accepting Unicode code points is more generic. I guess it depends on your environment. If every caller will combine it with a UTF-8 decoder, you might want to include it.

I meant something like a standard component that everybody uses (or a selection of components that vary here and there but are interchangeable in general). A catalog of such components; a - well, I'll say it - a software factory that builds them, a fast one, a slower but very detailed on errors, one that accepts single bytes, one that takes a pointer to a string, and so on, all different, yet all working in the same way as far as the main task is concerned.

Reply on news.ycombinator.com