Duktape: an embeddable JavaScript engine
duktape.org
duktape.org
I've lamented for a while that, while JS engines are flourishing from a speed perspective, there has never been one as small or embeddable as Lua.
Now we have JS as embeddable as Lua, and nearly as small (it probably can't be quite as small because Lua is a simpler language).
Now the only thing Lua has on JS, implementation-wise, is an implementation that is very small/embeddable and very fast (LuaJIT). LuaJIT is also about as small as a JIT-compiled interpreter can be.
But it looks like Tessel is working on filling that gap as we speak -- they are working on targeting LuaJIT for their JavaScript implementation http://blog.technical.io/post/102381339917/a-new-engine-for-...
Well, and a simpler, more regular syntax. Simpler semantics. Real coroutines. Tail call elimination. Multiple returns. Better iteration syntax and protocol. Better lexical scope. Operator overloading. A well-established FFI. No weird variable hoisting, nasty implicit conversions, semicolon insertion, etc.
But, sure, aside from those things, Lua doesn't have anything on JS as a language. (Of course, as an ecosystem, JS is miles ahead.)
EDIT: typos.
Not everything has to be strictly on-topic to have merit, especially since we've already derived from the topic (from a specific embeddable JS engine to the comparative merits, implementation wise, of JS vs Lua).
I said "implementation-wise" (it's even in the snippet you quoted).
The major issue to me is time. I know quite a few languages, but have only mastered a couple. JS is everywhere and the browser makes sure this will be the case for a long time.
The issue is that these advantages do not outweigh the ubiquitous nature of JavaScript. If I already need to learn all of those JS quirks, why learn Lua too?
Also, lua is a nicer language, and if you have the choice, you should chose it over js.
Sorry, but I prefer the nasty language with 0-indexed arrays.
In my case, because Awesome is a nice window manager.
Although on this theory, I should really have learned some Haskell by now. And yet. Every trip to the xmonadrc is a fresh glimpse at the depths of my own ignorance.
I'm not sure I follow. What is the distinction here?
I'd still prefer programming in lua to programming in javascript where possible. Honestly, the only advantage of javascript outside the browser are the hordes of people who already sort-of know it (and often don't want to learn anything else).
[1] http://www.reddit.com/r/programming/comments/badl2/luajit_2_...
Targeting the bytecode directly will not buy much, it is very close to Lua itself, semantically. The LuaJIT parser and bytecode compiler are also wicked fast.
Furthermore, IIRC, the JIT works best with idiomatic Lua code, compiled to "kosher" bytecode by LuaJIT itself.
We'll see how it turns out.
————————————————————————————————
Accepting sub-optimal performance in exchange for getting a JIT, garbage collector etc. for free seems like a good deal in many cases. Note that Pall estimates a development time of multiple years (!) for the optimal solution.
64-bit LuaJIT can only use memory allocated within the first 2 GB. Mainstream desktop systems consume more and more memory so by the time you fire up your LuaJIT app there might be (next to) no memory left within that range.
32-bit LuaJIT can use the first 4 GB but again sooner or later desktop systems will leave little there too + 32-bit code is already deprecated technology on the desktop.
Makes me damn sad really. Unfortunately making LuaJIT capable of using the full address space requires a major rewrite.. one which will likely never happen. LuaJIT's author has moved on to other projects and LuaJIT is now in maintenance mode, getting nothing but small fixes.
Seriously if you are a Mike Pall-grade elite low-level hacker (I am talking to all four of you ;) ) would you please continue LuaJIT development?
I don't know why you don't get the full 4GB range, though. If anyone's interested in digging through the header for the exact format, it's here: https://github.com/LuaDist/luajit/blob/master/src/lj_obj.h
Other applications won't affect your own process's address space.
Also, it's worth remembering that the 2GB limit only applies to Lua objects. Objects allocated through the FFI API aren't limited by this. So any pure data structures which don't contain Lua object pointers can be allocated with simple malloc(). It'll be loads faster that way, too.
Right now I do not feel comfortable offering a LuaJIT app for download because it may crash immediately with an "out of memory" error when the user has some memory heavy app running in the background.
Is the idea if you have a desktop app executable wrapper and you could include this engine to start building desktop apps in JavaScript?
I don't think I have a clear understanding of this type of use case or how this will be used, please share.
The only thing I can think of is you could use this JS engine allow folks to code in JS, instead of C/C++, that actually interface with C/C++ APIs? Examples anyone?
Embeddable, small-footprint JS allowing the masses to script things with sounds pretty useful!
- In a game, write quest logic in the scriptable language so that game devs can tinker with quest design without recompiling the game binary.
- "plugins" (usually UI plugins of some kind). Games like Elder Scrolls Online and World of Warcraft allow for 3rd parties to write UI extensions in Lua.
- Any other logic that you want to run and frequently modify without modifying the binary.
There are other great uses-cases that I can't think of right now, hopefully others can help.
- You already have working JS code that you don't want to rewrite for a new platform
It also greatly simplifies logic because you can do pattern matching and filtering (similar to Erlang) more concisely than in C.
I think that coroutines are on the verge of replacing callback hell so we'll start to see patterns from Go begin to show up in other languages, perhaps embedded in this way. The browsers dragged their feet for years by not implementing generators and yield (in ECMAScript 6) and set javascript back immeasurably, so being able to go around them with embedded libraries will be huge.
In a more generic sense, since 10% of a program causes 90% of the slowdown, there is little speedup to be gained by using C in the real world, especially if it's mostly using system calls that are already optimized. Some examples of this are how a high level language like php has succeeded in the server world even though it's not particular fast or elegant, and how Objective-C is often still slow even though it's low level (because programmers sometimes do too much with their "bare hands" rather than building upon heavily optimized libraries like C++'s Boost). I think this explains the rapid adoption of Swift, which is basically another high level language in place of a more painful low level one but providing similar performance.
Low-level (like, just about assembly, maybe slightly higher) workloads that require high performance or are annoying to do any other language.
What is C++ good at?
All of the stuff C can do, but also bolting on a weirder type system, generics, metaprogramming, and a whole bunch of other stuff. It's fast, but rapidly becomes a festering tumor of functionality.
It's a katamari of design features, and even though it exposes a lot of awesome high-level abstractions the fact that it still allows working at the C level prevents it from being compelling.
What about JS?
Javascript is dynamically typed, which is terrible in many ways but bloody useful for quickly getting shit done. It is a functional language with prototypical inheritance, and so naturally lends itself to high-level programming...with regexes and functions as first-class objects, and a really pretty simple type system.
It sucks at high-performance stuff, but it allows for a lot of clever and very brief code.
So, if you want high-performance and the power of high-level abstractions, it makes sense to use C and JS--each a fairly small language well-suited to its problem domain. C++ would require too much annoying work on both sides of the problem.
A good workload for this would be: in-game scripting, network message transforms, simple command-line utilities, game engine writing, numerical library writing, asynchronous IO processing..basically, any application where you can easily separate out the low-level primitives from a flexible high-level solution description.
~
Then again, for all of what I'd said, I think you might as well just be using Erlang or Elxiir. :)
Did you coin this phrase? It's a fascinating, evocative, and quite possibly accurate description of many of the systems I find most frustrating. In any case, thank you for bringing a smile to my face today :)
It seemed like it fit.
For embedding, you need some small language with small runtime. Something like Scheme/Lisp/Lua would work. Something like Python is not suitable. Lua is popular in the gaming scene because of its small size and its well designed architecture for embedding. Looks like Javascript is getting there.
This is a wild overgeneralization. Civ4 used Python as its embedded scripting layer to great success.
(Me, I just use C# as a scripting layer inside of a C# application. On desktops, the content C# is compiled or read from cache at runtime, but on other platforms I can include the scripting libraries as project references and have them incorporated into stuff like Mono's AOT linker. http://github.com/eropple/exor )
There are three concerns that make an embedded language good:
1) Lockstep multiplayer. Each player has a full copy of the gamestate on their client. Every action performed against the gamestate is done in sync on every machine in a deterministic fashion and order. This means that any custom code must be written with preserving sync in mind - something as simple as rand call on the clients would break sync. So Lua allows the engine developers to properly restrict the users to only use libraries and language constructs that will respect the sync (it fails at this, but it's a goal and Lua helps with that).
2) Code from peers. Users develop code for other users - widgets that help on the user-interface and games for each other to play. A sandboxed environment is a must.
3) Graceful failure. If a widget fails it doesn't bring down the whole engine with a segfault.
Another note: Try to do fib(33) in duk, it takes 9s, while lua interpreted mode takes under 2s. So no doubt it's still got a long way to go in terms of performance.
Personally though I prefer the libuv APIs over the node APIs.
shear - cut the wool off (a sheep or other animal).
You may not have heard of it, but you likely used it.
https://github.com/judofyr/duktape.rb
I opened a PR update it to 1.0.
Compiled through emscripten to build a JS interpreter in a browser... at only 1.5MB!
Disappointed mainly because the fact of no platform dependencies and two simple files would have made it a very very convenient library even from non-C.
Look for "BEGIN PUBLIC API" for the stuff you'd have to wrap. Stop at "END PUBLIC API". Nothing there should be that weird looking.
Also like a lot of these other "lua alternatives", it's interesting to note development seems to be driven mostly by one guy... which if you want to use it in a project brings up the question: "What if they get hit by a bus?".
If I ended up having to maintain duktape I'm confident I could understand and improve the code with a little rampup time. This isn't rocket science. The hardest problem is finding time and funding. If a company depends on the project they have incentive to provide those things.
I suspect if you're familiar with Lua then you'll be almost immediately at home, although people new to both might struggle with the idea of stack-offsets/indexes.
https://code.google.com/p/tiny-js/
I started using it for a small side-project but had to put it on hold because of issues elsewhere in the stack.
From a cursory glance at things, Duktape looks way more advanced, and being written in pure C should make it much easier to embed. I'll definitely give it a try when I have time to look at that project again.
That said, the TinyJS code is extremely simple, so it makes a potentially good starting point for extending, if you want something JS-like rather than a pure JavaScript engine.
Also the portability of Duktape impressed me. I was able to get a test up and compiled in cygwin within a few minutes.
Edit: I see the requirements are right there.
You can run Duktape with 128kB RAM, and to some extent with 96kB, see e.g.: http://www.slideshare.net/seoyounghwang77/js-onmicrocontroll...
First of all duktape written in C not C++ I hope this is a good point for you as it was for me :)
Not everyone seeks speed, for me, simplicity and ease of use beats speed any given day, after all I'm not after complex computations and fib that loops for ever :) one file to include and pretty sure it well compile in many platforms to get a full working Ecmascript E5/E5.1 how awesome!
Compilation time, don't know about current SM compilation time but I tried once to compile V8 and I still feel sorry for doing that to my poor old machine :)
Memory management, I'm not sure how, but duktape's GC is amazing.
Last and the most appealing point for me "unfortunately no one mentioned that yet" is documentations, I don't mean just api, please visit duktape.org and see how beautiful well written everything there, Sami "the author" put a great effort not just to write duktape but it's documentation too, api, usage, comments, specifications, examples ... and best of all it was clear and very easy to follow.
The only thing I hope is more people join to help Sami maintain the project, currently the man is very active and responsive, but I think the project will expand in the next few weeks with more and more users, so I guess he will need some help then :)
Source code at https://github.com/espruino/Espruino
Espruino boards at http://www.espruino.com/
This library is great:
On the simplicity side:
Only include two files, no linking, no shared objects, no cruft.
On the feature side:
The functions are simple, the usage-pattern is familiar (alloc, use, free) and the code is fast (written in C, which _almost_ makes it fast by default).
Great library, will probably use it when I need to write some more scrapers in the future.
I hate to be that guy, but [Citation Needed].
Most language implementations are written in C and most language implementations are quite slow, once you include the giant slew of hobby, toy, and less successful languages. Remember, Ruby, Python, and early versions of JS interpreters were all written in C too and are or were dog slow!
Duktape does have a section on performance[1], but it's basically just a guide to things you have to do to not make it extra slow. I didn't see any reported benchmarks.
This doesn't mean Duktape is bad. Simple+slow is a perfectly good trade-off for many real-world uses. Embedded scripting languages tend to spend most of their time in the FFI, not executing code, so perf doesn't matter as much as maintainability.
But it's really dubious to just say, "well, it's written in C so it's probably great." It's using ref-counting and linear scanning or hash table lookup for property look-up. Perf is likely not great.
Just as a point of comparison, my little hobby language[2] is also written in very simple C, but it's carefully designed to be fast too. Property look-up and method calls are just pointer + known offset. GC is tracing. Because of this (and other stuff), perf actually is good[3], typically better than Python and Lua and on par with Ruby. (With the usual caveat that all benchmarks are bad. :) )
[1]: http://duktape.org/guide.html#performance
[2]: https://github.com/munificent/wren
[3]: https://github.com/munificent/wren/blob/master/doc/site/perf...
That's not a fair comparison. Speed is relative. Compare them to implementations written in other languages.
Languages that have decent implementation speed are a rare outlier compared to the hordes of slow implementations out. That horde of slow languages is wide enough to also encompass slow implementations in other languages too.
Here's Lua: https://github.com/lua/lua/blob/master/src/lopcodes.h#L13-L2...
https://git.allseenalliance.org/cgit/core/alljoyn-js.git/sum...
Why not just embed an API and call it SQL.
Luckily Javascript has a great ecosystem, so there are non-compliant subset engines targeting tiny devices. What other (higher level) language scales this way? :)
Integrating duktape into my project was literally just including the C file. It added just a couple hundred KB of overhead.
Compare that to SM or V8 that require massive custom build systems and use up to 10mb of ram for a simple repl.
Duktape: an embeddable Javascript engine,
with a focus on portability and
compact footprint.
Without that, the title is effectively click-bait, requiring a click just to find out what it's about.Edit: I see it's changed - thank you mods, or contributor.
https://github.com/robertkrimen/otto
Here's an example Go app using otto for React server-side rendering: