How the embeddability of Lua impacted its design | Hacker News Reader

How the embeddability of Lua impacted its design

queue.acm.org

104 pointsRickasaurus18 comments

I initially picked up Lua because it was so easy to embed ( also it was small ). Recently I have seen it used more as a stand alone language, but it really shines when you need to add a scripting layer to an existing project. I was making a GUI for a control system and I exposed all the drawing primitives and control points to Lua and was able to get a very fast prototype cycle working.

One of the classic FAQs about Lua is:

"Why doesn't Lua have unicode support?"

I mean, it does, but in the same fashion as ANSI C: you need to either pick a convention for multibyte characters or use a third-party library.

And that's not an accident. One part of Lua's embeddability design is that the standard implementation is written in pure ANSI C, and pure ANSI C ... doesn't support unicode.

Standard C has wchar_t, which is used for Unicode on platforms supporting Unicode.

wchar_t should never be used in any new code

Just out of curiosity: What is its replacement? If I remember correctly it began its life as the char type for multi-byte character sets and later found use for UTF-(16|32) code units.

Should Unicode strings forcibly be UTF-8 in all programs and use char again (thus suffering from the same problem that Python 2 had where bytestrings and Unicode strings weren't sufficiently distinguished, often leading to programmers using the former for text)?

Or should you use short/int arrays instead and have no support from the stdlib at all for handling them (not that the stdlib is particularly good at handling text)?

If you're using UTF-8, use either uint8_t* (preferred) or char. If you're using UTF-32, use uint32_t. If you're using UTF-16, don't.

As far as library support goes, there are a few good Unicode libraries for C, most notably ICU4C.

wchar_t is implementation-specific.

Sure. char is not specified to support ASCII either but it's not really a problem in practice :)

If you start avoiding all the things that are not fully specified by the standard, your C programs will end up doing very little indeed.

Now, it might be that some environments targeted by Lua have very spartan C runtimes: consoles. Anyone have a pointer to the C runtime docs for Xbox 360 and PS3?

I don't understand how writing it in "pure ANSI C" prohibits Unicode support. Python, for example, is written in "pure ANSI C" and does support Unicode. Though doing so correctly and completely is hard, and requires code and data which bulks up what is otherwise a compact embeddable implementation.

Basically, it's as you guessed: Lua's string implementation is pretty much a transparent skin over C strings. The main difference is that Lua checks the boundaries and collects garbage.

It's a real turnoff for lots of people. In terms of "things new to Lua find unattractive", it's second only to 1-based indexing.

Hmm, the majority of this is about implementation, not semantics. I like looking at choices made in the Kyoto family of lisps versus most other CL implementations as good examples of tradeoffs when embedding.

Could you elaborate on that? I think this is just a case of the Lua authors being too academic to let implementation details mess with the semantics too much. They say that there is lots of tradeoff involved but they only mentioned using simpler dynamic typin instead of sophisticated static typing and a very barebones module system.Keep in mind that they don't have to compare with other existing implementations like the Common Lisp example though.

That said, Lua has evolved considerably over the years, specially the embedding API (I belive it has been completely redesigned more then once). Sometimes thery added changes that made the API simpler (for example, globals used to be accessed using special functions, now they use table lookup, as mentioned in the article). Other times, things were originaly limited due to the implementation, but they got "cleaned up" in latter versions (upvalues gave way to proper lexical scope, "tags" became a more general metatable system, etc).

I'd love to read a blog post or paper on this topic.

A fairly detailed paper on Lua was published in HOPL-III in 2007: http://www.lua.org/doc/hopl.pdf

It just occurred to me that a language could be made embeddable using eval() alone, with okay performance. We just need to take a page from SQL's book and introduce "prepared statements" for eval. A string of high-level code with placeholders could be parsed only once, and converted into an object you can "call" using different values for the placeholders. I'm pretty sure this isn't a new idea, though.

Somewhat OT, I can't believe I'll end up dropping out of college before I take a class with Ierusalimschy. I've been told he will usually wear a bermuda and flip flops to class but unfortunately those are few and far between for undergrads, only Compilers that I've heard of, while most of his work is with grad school.

As someone that has recently moved to Rio, I have to confess I'm baffled at how people manage to not wear a bermuda. Its so hot here!

Other then that, I think he gives some lectures on semantics every once on a while. The compilers class is only rarely offered since so few people take it :/

Try again next semester. Taking compilers is mandatory for comp. sci students, so they will have to bite the bullet and take it eventually, even if scared of Roberto (somewhat with reason; the course is easier than Coursera's Compilers course, but still a hard course for the undergrads).

Reply on news.ycombinator.com