A Lua 5.3 VM and compiler written in Go
github.com
github.com
https://github.com/yuin/gopher-lua
https://github.com/Shopify/go-lua
Note that Lua's versioning scheme is not like most other languages. Lua 5.1, 5.2, and 5.3 are not the same language. Though most code will still work across those versions, the Lua developers are more aggressive at adopting new features and killing old ones, even for core functionality. 5.2 dropped function environments in favor of lexical _ENV, which was a huge change for people implementing sandboxing, reflection, module frameworks, etc. 5.3 added an integer type whereas previously all numbers were floating point, and because Lua is dynamically typed you can't move an existing body of Lua code to 5.3 without some careful attention. Similarly, the APIs also have incompatible changes that while mostly simpe to deal with require porting attention.
This more aggressive style of stewardship is what has made Lua an amazingly elegant and powerful language, but reduces it's attractiveness for large, monolithic, long-lived frameworks unless they're prepared to periodically upgrade languages and migrate their user base. Since Lua is more typically used for extension and embedding (i.e. not monolithic, pure-Lua projects), this is less of an issue.
I was explaining to a colleague the other day that the reason you don't hear so much about Lua is because choosing Lua isn't a decision that dominates a project. A project that heavily uses Perl is a Perl project; one that uses Python is a project. For the most part, you don't build Lua applications; you build C, C++, or w'ever applications that use Lua, even when 90% of the code is in Lua. This is a reflection of the carefully crafted API and language semantics. But IMO it's the biggest reason why Lua hasn't yet had a break-out moment, despite Lua 5.1, 5.2, and 5.3 being (IMNSHO) far superior languages to JavaScript, Perl, Python, Ruby, or other similar dynamic languages. Lua is a better language because it aggressively improves itself. But that evolution is too rapid to support many large, marquee projects.
That said it was trivial compared to the _ENV change for 5.2
But people were still anxious, even gung-ho Lua users. Whether or not justified, people and companies contemplating large projects are especially averse to that sort of environment.
My open-source project experiences notwithstanding, before I left my day job a few months ago we still hadn't migrated our 10+ year-old project to 5.2, let alone 5.3, even though LuaJIT wasn't involved, and even though lua_callk and lua_pcallk would have significantly simplified the libevent async I/O integration. In a corporate environment planned obsolescence is a hard to sell, even when it's objectively the better choice.
The LuaJIT (http://luajit.org) is another reason why Lua is very popular (although it only officially supports Lua 5.1). It’s a very fast implementation of the Lua VM. OpenResty (https://openresty.org/) is a very popular framework that allows Lua scripting on top of NGINX and LuaJIT, and it’s the same stack that Cloudflare uses. Basically over 5% of the world internet traffic goes through Lua, but only a few people know it.
This is the same stack that Kong (https://github.com/Mashape/kong) also uses, probably the most popular OpenResty-based application. Lua on top of LuaJIT can be extremely fast especially if you leverage FFI.
Lua is an extremely simple language which can be pretty much embedded anywhere, and the LuaJIT just does a great job at making it extremely embeddable and fast.
Disclaimer: I am a core committed at Kong.
Nothing to see here, really. Just remaps basic Lua syntax to Go without actually _implementing_ what makes Lua useful.
I'm aware of Lua and JS interpreters, some fully native, some made of binding against existing interpreters. Native JS interpreters have my preference right now: easier to build/maintain, I can bear the loss of perfs, and JS seems easier to sell than Lua (remember it's entreprise stuff).
Does anyone have some insight or experience in that area and care to give feedback to a fellow HNer? And did I miss other scripting languages?
Other options are TKL, but the syntax is a bit unorthodox. Or Python, but it is much bigger, and harder to embed. JS is not a great fit, I would say, mostly because it is comparatively HUGE and not particularly optimized for embedding.
There are big drawbacks for using Perl though. It's appallingly bad for writing OO code (objects in Perl are really just hashes / maps) so larger code bases or complex code can quickly become a pain to maintain. While it has one of the better approaches for comparing data of different types (imo), Perl still has more than it's fair share of hidden traps.
Ultimately though, it's a better language than many give it credit for even if it is considered old fashioned these days. So if languages like Python and Javascript are to be considered then I think Perl deserves a mention as well.
As for implementations, the standard Perl interpreter can be linked to which makes it very easy to embed into other programs.
I've written far more than my fair share of Perl XS code over the past nearly 15 years. I still use Perl and the last time I wrote Perl XS module was just last month. I could go on for pages about the negative aspects, without having much to say about the positive aspects.
But here's a very concrete reason why nobody should contemplate embedding Perl: I filed this bug report (Thread race in dist/IO/IO.xs) over two years ago:
https://rt.perl.org/Public/Bug/Display.html?id=125685
Non-threaded Perl embedding may be more common, but ultimately you don't want to be embedding any interpreter that has shared state, especially if the fact of this shared state leaks into interfaces or breaks the interface contracts. Python is also still problematic in terms of not being able to completely erase vestigial assumptions of one interpreter per process. Neither Perl nor Python were originally designed for embedding, and even their extension APIs were bolted on. While they've both been improved over the years and work well for straight-forward C bindings, the fact that neither the languages nor implementations were originally designed for embedding eventually comes through and cause headaches when you want to do anything sophisticated--sometimes even when you're not doing anything sophisticated.
[1] The only significant use of Perl in the open source world that I'm familiar with that embeds Perl this way (multiple instances, multiple threads) is Apache mod_perl. But for other reasons mod_perl creates and initializes all Perl instances from a single thread. This is why the bug went undiscovered for so long after it was introduced, and why there's little impetus to fix it.
Threading is an interesting point. It's worth noting this issue exists with quite a few scripting languages though and the way Apache gets around it (eg with mod_perl, which is garbage by the way) is by forking rather than threading (I mean you can obviously run Apache with threading if you can guarantee all your XS code is thread safe, but it's safer to just prefork)
I would really love a fast, lightweight stripped down "C Flavored Syntax" dynamic language to use instead of JS or Lua but I haven't found either yet.
Not sure if there's an easy fix to that apart from using `null` instead of empty lists.
One thing I noticed is that the parser is hand written. Which is interesting. I always preferred to write my parsers in Yacc and Go actually has Yacc as a built in tool (although know Lex).
Edit: I went through the list someone else posted and only one of the implementations actually has a Yacc grammar file in it.
Interesting. Did they combine both a lexer and a parser into one tool?
lex,flex, and re2c are typically standalone tools (lexers) that tokenize input, whereas yacc and bison are parsers that parse those tokens into ASTs (usually).
No. They give you an interface definition for the lexer and you can implement the methods however you see fit as long as it matches the interface.
Which I found annoying as there are some tedious parts to writing a lexer as soon as you have a language requiring multi-rune tokens with common prefixes.[1]
Apparently the back story is that go needed a parser generator but they didn't need a lexer so they only built just the one out of necessity.
I read some places that they would welcome pull requests for a lex/flex equivalent.
[1] A `rune` in GoLang is like a `char` in C only it stores multi-byte characters where as in C it is more analogous to a single `byte`
A rune is just an alias for int32. But it's context usually means unicode chars which are indeed multi-byte.
You can still use byte arrays (or "slice" if you're writing idiomatic Go, but that's another tangent again) too though.
Also "byte" in Go is another alias, but for int8. But that shouldn't come as a surprise to anyone :)
That said - many languages I dabble in are variants on Forths or Lisps where parsing is easy comparatively.
For posterity, it's moved to https://godoc.org/golang.org/x/tools/cmd/goyacc
The statement that he could easily implement coroutines via go seems like oversimplification or lack of problem domain understanding.
For someone with a lack of problem domain understanding, could you elaborate on this? :)
> Anything to do with the OS or file IO is not provided. Such things do not belong in the core libraries of an embedded scripting language (do you really want scripts to be able to read and write random files without restriction?).