Luerl - Lua on the Erlang VM
erlang.org
erlang.org
1. I want to experiment with implementing an imperative language with mutable global state on top of Erlang. A goal was to do it in straight Erlang without extensions.
2. I am giving a talk on implementing DSL's on Erlang at the next Erlang Factory in San Francisco in March, http://www.erlang-factory.com/conference/SFBay2012 (be there, it's going to be good, the whole factory I mean), and I wanted to be able to give practical experience from implementing a real-life imperative mutable data language. Lua is a nice, small, relatively clean language and is a good choice for this. And it is actually in use. I was also considering PHP or Javascript but they are definitely not small and clean.
3. Lua is an interseting language in its own right and has some cool ideas. You can get quite far with a few simple primitives. I don't know if these would be transferable to Erlang.
4. I like implementing languages.
While erlua is definitely interesting it is "just" an Erlang interface to a standard Lua system and not what I was after. I know that this idea is being used in some commercial systems so it is definitely a viable option.
Whether this is of any practical use I don't know. For doing large amounts of Lua execution then a solution like erlua is probably better as the straight lua implementation is faster than luerl can ever be; doing mutable data in an immutable environment is not free. However, the interface between Erlang and Lua in luerl is definitely faster so for smaller amounts of work it may be better. There is also the benefit of while there are many languages there is only one system running them. Could lead to easier maintenance.
This only a first version and there is much still to do. I am experimenting with different ways of implementing Lua's mutable data and I am looking at compiling the Lua code down to native Erlang machine code; the current version is an interpreter. There are also some language features left to implement. More libraries would help.
That is the status as it stands. My goal is to have a system where people could take their Lua code and just run it in luerl.
I started using Lua because I needed to gather logs from many different systems and build timelines around the events to identify patterns across all the aggregated data. The logs were large, and the metadata was often very complicated.
I looked at several SQL-like solutions, and at Perl extensions on CPAN. After lots of googling, it finally occurred to me that I might need a programming language that had been designed with efficient data description as a high priority. I was surprised that a streamlined "little" language like Lua fit my requirements [0]. I was expecting to have to learn something big, complicated and esoteric.
I needed my resulting datasets to be portable between various Unixes, and fairly standalone. Because of the long load times, I didn't want to use a tool that required any more processing steps on input than absolutely necessary. While serialized Lua tables are more verbose than CSV, they do compress well. They are also valid Lua programs. They can include metadata of arbitrary complexity, and even turing-complete snippets to modify expression of the data as it is loaded.
Salvatore Sanfilippo (antirez) had many languages to choose from for his "data structure server". The next major release of Redis will have a Lua interpreter. [1]
I don't know Erlang well enough to guess -- is a valid Erlang program an appropriate way to store a large amount of polymorphic data? Is Lua perhaps a better way to approach this problem?
[0] "Lua as a data description language (Dimitris Papavasiliou)" http://lua-users.org/lists/lua-l/2004-06/msg00028.html
[1] "Short term Redis plans" http://antirez.com/post/short-term-redis-plans.html
I don't know about embedding it in Erlang, but anecdotally, it works quite well as a data dump format.
1. All network I/O is non-blocking, even if you write code in a blocking manner (i.e. no callback programming when writing scalable servers)
2. Erlang's processes are green threads, so you get massive concurrency for free (i.e. no cooperative thread programming when doing large-scale multi-tasking)
3. In an OTP environment, code will be executed as part of a supervision tree, so it will be very difficult to crash the interpreter (i.e. no strict need for error- and exception-handling code)
So while this interpreter will probably be quite slow for single-threaded tasks compared to the standard interpreter, there will be benefits if you want to run a lot of Lua code concurrently in a networked environment.