Elk: A low footprint JavaScript engine for embedded systems | Hacker News Reader

Elk: A low footprint JavaScript engine for embedded systems

github.com

181 pointspjf89 comments

This implementation is nuts. A decent chunk (but still very small subset) of ES5 in a single file, under 1400 lines of very readable C code. It includes a mark-and-sweep GC and an FFI.

It doesn't have an AST or a bytecode VM. It just interprets directly off of source code.

Take a look: https://github.com/cesanta/elk/blob/master/elk.c.

The first time I scanned through the file and thought there was nothing there. It does an entire implementation in the same amount of code that most implementations do a parser alone (but yes, it implements way less than a 100% spec compliant parser does). This implementation really sets a new bar for me in terms of compact-but-readable language implementations.

Separately, this isn't even Cesanta's only embedded JavaScript implementation. They also have: https://github.com/cesanta/mjs. mjs is a bit more complete and does have a bytecode VM and thus the source has many more lines of code and files.

You know, I'm seeing a lot of "why JS and not X" in threads like this. and as much as I'm anti-js, there is a simple answer that's not all that technical: It's what the author likes and wanted, so they built it.

At the end of the day, that's all that matters for it to become reality. if you want something like this but X language, I'm sure there are similar projects, or you can build one yourself.

I have no use for this and don't like JS, but I appreciate and respect the efforts put into this project, especially with the clean implementation.

I love JavaScript and Typescript. They’re fun and easy to read and write, and there is little they can’t accomplish for the average use case. I haven’t dug nearly as deep into other languages, but I’ve dabbled with dozens, and Ecmascript 2020 is still my favorite language to code with.

The ecosystem is gargantuan, npm is great with pnpm, and all of the annoying parts of client side JS is made fun and easy with Svelte. With TS, CSS, and HTML (and Svelte bringing out their full potential), I can quickly make almost any app I want with incredible DX, comprehensive tooling, and next to no boilerplate. Every year it gets faster and faster, and when it’s not fast enough, there is WASM.

I would genuinely love to know what is so bad about it, and what I’m missing out on!

Personally, some of it is the language. the need to triple equals and all the type coercion. it's harder to get those wrong with typescript I agree.

another is the ecosystem. pnpm feels like a bandaid to a bad package management system . the whole 'node_modules' folder ends up being a polluted mess very quickly. I import a small library and I feel like I download half of npm registry with it.

any time I've wanted to do anything in the JS world, I have to download 30 different frameworks, that don't quite mesh well together. svelte+tailwind a year ago was a disaster to setup with rollup. had to switch to webpack which pulled in a bunch of other stuff and svelte broke.

I think the biggest problem is interop and all the translators and transpilers in a standard application. between Babylon, es6/7/8/next/jsx/TS/tsx transpilers. the scss, css, transpilers. media pipelines and converters. and half the time if you don't set them up just right. they don't play well together.

maybe it's just not my world. I'm a backend/DevOps/systems dev. and while many languages have similar problems, rarely do they have ALL of these at the same time.

back to the language, I feel like early on all the prototype overriding/monkeypatching/etc made for a very very messy language. I'm familiar it's gotten better, but all that cruft is there and still used often.

finally, while all the WebApps have been good, I hate how EVERYTHING is using JS on the web, even for useless things, or things that don't need to be JS. I hate how many sites literally are blank I'd you don't have JS enabled. it's also slowed down the web a lot. things take forever to load.

so I have many factors that contribute to make having a disdain for JS, both social and technical.

It's hardly fair to go to a thread on an embedded JavaScript implementation to criticize JavaScript by focusing on the JavaScript ecosystem. It's up to you to decide which and how you use a third-party package. If you adopt a dependency that's so bloated that it drags in half of NPM's repository them the responsibility of your personal choices lies solely on you and on you alone. I mean, you can fall for that mistake with pretty much any stack or framework or programming language which supports modularization and provides a package manager. Why single out JavaScript?

I didn't come here to criticize, if you notice my first post, I came here to tell others to not to, regardless of my beliefs. then I was specifically asked for more detail, so I answered.

and I single out JS because I like programming languages and learn and use most of them, JS suffers from this problem the most. anyways, sorry, I did not intend to argue over this. so I will refrain from continuing. I have my personal views on JS and I was trying not to detail the thread, seems I failed.

I'm with you that most of the criticism of JS is directed at 2011 JS rather than 2021 JS.

A few things that aren't great for my use cases:

- The language has its fair share of footguns. It's a much better language than most people give it credit for, and most of them are easily avoided by a good linter, but still doesn't feel as clean as something like Python.

- The ecosystem is great but it's hard to separate the wheat from the chaff. node_modules with gigabytes of dependencies isn't even funny. I'd still classify the ecosystem as a big win for JS, but there are things that could be handled better.

- The standard library is awful and some parts are downright broken (e.g. dates). If I'm using C++, Java or Python I have at my fingertips very powerful data structures an import away. Many of those things are also available for JS, but they come as separate dependencies.

- Having the same language full-stack is nice, but client-side JS and server-side JS are different enough that there's going to be a context switch regardless. That point is true but somewhat oversold.

(You didn't say this and I'm not going to claim you did, but don't even get me started on how full stack JS evangelists try to sell you mongoDB-like datastores. There is a 99% chance the correct place to put your data is a SQL database.)

- It doesn't play nice with C.

- Doesn't lend itself to quick stuff outside the browser/web server paradigm. Things like OS scripting or small ad-hoc programs.

> Things like OS scripting or small ad-hoc programs.

I would imagine that you can run those in Deno (for example ) just fine?

I have no doubts that Deno or Node can run those fine, but their evented interfaces make everything harder than it needs to be, especially since those tasks often require reading and writing to stdin/stdout/files.

It's not a problem for larger projects, but very clunky in a script.

From what I can tell, Deno doesn't force you to use evented interfaces for I/O.

The problem is not Javascript itself IMO, the problem is that there exists a large portion of web-only developers with knowledge only about Javascript, who try to cram it down every corner and every problem space that they can. This leads to things like Electron polluting a space where Qt/wxwidgets/win32/GTK would be technically superior by producing small and native binaries, yet we get steaming messes of Chromium that swallow hundreds of megabytes of RAM and waste millions of CPU cycles.

In 2021, nobody likes programming efficiently in native languages anymore it seems.

That's not the problem. I did GUI programming for a decade before switching to web.

The reason that people write GUI apps in Electron instead of Qt/Gtk is because the latter is so far behind technologically that it feels like ancient technology. Electron has GPU-accelerated, declarative graphics in CSS, it has React, it has the web inspector, it has a scripting language with an extremely good JIT.

Qt/Gtk has not adapted to the state of the art, it has stayed static while the world has moved on. It's now 20 years out of date. That is why people use Electron.

Qt has GPU-accelerated graphics with the QtQuick scene grqph since 2014. Qml is a declarative language and quite easy to learn with Js based property bindings. It also has an inspector with Gammaray and many profilers (e.g. hotspot,heaptrack, qt creator profilers, kcachegrind, massif,...). The real reason why people are not using it more is the licensing LGPL3 is a bit too scary for some people and the commercial license to expensive

Non-uniform non-platform-widget based rendering of applications is not something I would call state-of-the-art. The overton window has only shifted to that recently. I would claim that a small statically linked executable displaying a native window containing native widgets is state-of-the-art, and would blow everything ever created in webland out of the water instantly in terms of resource usage and performance.

Since you named it explicitly - why would my application want a web inspector or CSS layouting overhead if the goal is to display a small batch of controls? This is exactly the mindset I meant when I wrote my post.

Red Hat and other Gnome-/GTK-adjacent people were on the JS bandwagon well before Electron was a thing. Most Electron developers have never actually made a good faith effort to try out JS development using GTK. How many Electron developers even know that when they interact with a vanilla Ubuntu desktop, they are using a window manager and desktop environment written in JS? I'd be skeptical that it was anywhere north of 15%.

If GTK had received a fraction of the attention and misdirected enthusiasm that went into bolstering the NodeJS ecosystem, things would be very different.

The set of features not supported is quite extensive: https://github.com/cesanta/elk#not-supported-features

  * No var, no const. Use let (strict mode only)
  * No do, switch, for. Use while
  * No => functions. Use let f =  function(...) {...};
  * No arrays, closures, prototypes, this, new, delete
  * No standard library: no Date, Regexp, Function, String, Number

Seems more "JS inspired" than actual JavaScript.

There is also QuickJS (by Fabrice Bellard) that I use and it's very fast to start and run js programs. QuickJS is a small and embeddable Javascript engine. It supports the ES2020 specification. https://bellard.org/quickjs/

QuickJS is also fantastic, but not anywhere near Elk in terms of size. Elk is ~59k (1300 lines) of C code, and has some limitations (like no for loops, for example) that I assume were traded off so it could run on things like the 2K RAM / 30K flash Atmel shown in the article.

QuickJS is small compared to say, V8, but quite a lot larger than Elk. Just the main C source file for QuickJS is ~1.7MB, and there's several other source files.

Just saying they seem to be in different niches.

This part of the code hints at what else is left unimplemented:

    case TOK_CASE: case TOK_CATCH: case TOK_CLASS: case TOK_CONST:
    case TOK_DEFAULT: case TOK_DELETE: case TOK_DO: case TOK_FINALLY:
    case TOK_FOR: case TOK_IN: case TOK_INSTANCEOF: case TOK_NEW:
    case TOK_SWITCH: case  TOK_THIS: case TOK_THROW: case TOK_TRY:
    case TOK_VAR: case TOK_VOID: case TOK_WITH: case TOK_YIELD:
      res = js_err(js, "'%.*s' not implemented", (int) js->tlen, js->code + js->toff);
      break;

So that's quite a lot of the language I guess, but it does say that it's a sub-set so of course that's fine. Very impressive code, it really shows that the developer has an eye to minimizing the memory footprint. I was fooled for a couple of seconds by the init call:

    char mem[200];
    struct js *js = js_create(mem, sizeof(mem));

Since it really "feels" like a function that creates a struct instance on the heap, but of course it does not: it's created inside the working memory passed in (so if it works, js == mem).

I write embedded code and the moment I see (data, sizeof(data)) in any kind of initialization method, I immediately expect it not to allocate.

Trust but verify, but also I'll feel mislead if it does since it's an extremely common pattern

The project appears dead though. It looks like the GitHub mirror has a couple PRs but no one is looking at them and it doesn't look like he's added any collaborators.

It might not be updated often but I find QuickJS to be quite stable. I've been building on it for years with no compatibility issues whatsoever. It's well written, and I don't think the fact someone hasn't made a commit recently makes it be of any less quality or usefulness. Fabrice works on many projects and is quite prolific in open source, he's probably busy.

It says clearly on the GitHub mirror that pull requests aren't accepted; there's a mailing list that you can submit patches to.

If something works, why does it need to be actively developed ad infinitum to be useful?

Because JS is an ever-evolving language. ChakraCore has been "abandoned" by MS (technically transitioned to community ownership) for less than a year and it's already behind on ES6 and ESNext compatibility.

I've used QuickJS but when I got stuck on an issue (stack overflow in win32 because of passing by-value) I got no help from the tiny community on the mailing list and had to abandon the attempt and go back to (overweight) V8.

If ChakraCore is behind on ES6 compatibility, that has nothing to do with its recent abandonment. ES6 was finalized over 6 years ago (not to mention the time that was available to work on supporting it concurrent with the development of the actual standard itself).

Well if we are mentioning different projects, there is also Espruino which has alot of support and drivers. One of rhe neat things about it, is that it gives you a "browser" like console to interact with your javascript and has commands similar to Node.js. https://espruino.com

I implemented once an Espruino target for webpack to enable a git workflow, npm packages, and transpile modern JS features to ES5, as supported by Espruino (targeting an esp8266 specifically).

Boy oh boy I did not expect to run into memory issues so quickly. Including async/await increases the size of the build so much the esp just dies.

But most npm packages worked just fine, including lodash ( a life saver), as long as you’re willing to work with the super tiny memory budget and the difficulty to detect memory leaks.

Using an esp32 with 8mb of PSRAM yielded much better results.

Good times.

Well it's designed to run on NRF processors with just 64KB of RAM and as little as 128KB of flash, which last a week on one CR2032 battery.

You might have been over doing it! /s

I regularly develop intensive apps in Espruino for the ESP32 (I mentioned in another thread how I pull in exchange data directly using HTTPS, and process it on the TTGO Watch 2020), but it's all hand crafted code. Maybe you should consider a small Friendly Elec board and node...

...and yes PSRAM is super awesome! :)

> You might have been over doing it! /s

You’re completely right! Fed up with Arduino C, I tried to bring the entirety of the web toolchain and related dev patterns to the embedded world. :’)

What you’re doing sounds super cool! Is it just as a hobby or your day job as well?

Do you have any tips on debugging long running memory leaks? (basically the board crashes after a few days running continuously, which were my specs)

Nowadays I’m back to Arduino C and the M5Stack/M5Paper boards, it’s a bunch of fun when I find time for it!

> What you’re doing sounds super cool! Is it just as a hobby or your day job as well?

It's kind of a serious hobby. I enjoy it.

In Espruino the memory leaks are always almost the variables. So you can do say process.memory() to see if indeed your memory is decreasing, and if it is, then you can E.getSizeOf(global) to see which variable it might be. It's still javascript at the end so, standard techniques apply.

Yeah I like the M5Stack boards, they're quite interesting. I use the their StickC. The orange one. It does take time for sure!

Fabrice Bellard is making us, average programmers, look bad.

Has been since the 90's. QuickJS is just one of dozens of incredibly complex projects he has worked on, relative to the abilities of an sole programmer.

`ffmpeg`, `QEMU` and `tcc` also want to have a word ¯\_(ツ)_/¯

QuickJS' base memory usage is somewhere around ~200KB, it's not really targeted at embedded devices. When you have 8-64KB of RAM you're better off running Elk, Duktape in low memory mode, or other engines developed specifically for this purpose like Espruino/JerryScript.

Names are (not-so-) strange attractors. To me, "Elk" in this kind of a setting is "Elk Scheme - the Extension Language Kit"! [1, 2]

[1]: http://sam.zoy.org/elk/ [2]: http://www-rn.informatik.uni-bremen.de/software/elk/

Elk Scheme has been around since nineteen eighty seven. Given the power of embeddedish devices these days, I wonder if Elk Scheme would be a useful embedded language implementation in 2021? Sadly I don't have the cycles to experiment.

Has anyone here used scripting languages on a microcontroller? And if so why? For user-scripting? I can't really think of another practical use case but there are multiple implementations of pretty high-level languages out there so someone must be using them.

You'd have to build some hardware abstraction to use with the scripting language anyways at which point you could just use the language you wrote the abstraction in imo. Is it so you can have inexperienced (cheaper?) devs do the maintainence? For all the embedded projects I worked on the high level program logic wasn't the thing that took the most effort.

That is like asking a Spring Framework developer using all those juicy annotations the reason they do it vs rolling their own in pure 2006 Java code.

Knowing how to do it vs using a higher DSL is always going to be nicer especially to gain new users. Look at the sega genesis/mega drive SDK. Many 2020+ Sega Genesis/Megadrive games are built using it. It is C. Why not use 68000 assembler like every developer before now? Documentation, ease of use, community, cross domain developers can pick it up easier...

Same thing applies here.

Too bad they didn't write it in Rust.

I believe C is much too fragile for interpreters to be written in them, especially for high-usage scenarios like the web.

As mentioned a few days back:

Enumerating and analyzing 40 non-V8 JavaScript implementations

https://news.ycombinator.com/item?id=28613673

Why?

> Instead of writing firmware code in C/C++, Elk allows to develop in JavaScript. Another use case is providing customers with a secure, protected scripting environment for product customisation.

Right, but why would anyone want to do that? Who thinks that JavaScript is a good choice for this?

Why is JavaScript a bad choice for this?

Many reasons occur to me, but the top few would be:

* Lua already exists and is better suited to this

* JS is a relatively complicated language to evaluate

* JS requires a large amount of dynamic allocation

* JS isn't really the first thing (or in the top 25 things) I would pick when developing on a platform where I wasn't already stuck with it

- JS requires a large amount of dynamic allocation

One of the top listed features of this interpreter is a fixed memory footprint

- JS is a relatively complicated language to evaluate

In developing this interpreter they eliminated some of the languages features to make it simpler.

> In developing this interpreter they eliminated some of the languages features to make it simpler.

That means you can't run general-purpose code safely, which means you should probably write/rewrite/adapt it, which means you might as well use a different language.

I'm a JS developer, but the parent might have a point here. Why run something like this in production when you're likely to end up in unexpected situations? Either a runtime is compliant, or you're going to have a bad time.

The project is cool, but I wouldn't use it as an example for what JS can do.

> One of the top listed features of this interpreter is a fixed memory footprint

In embedded systems that is often a necessity but calling it a feature instead of a constraint is a good idea.

I imagine it's nice for things like a product that allows some amount of scripted control by end users.

People who know javascript

And can now run in a tiny engine!

People that grew up with BASIC, z80 and 6502 Assembly and know C isn't the answer for everything.

The example with ESP32 is quite, the hardware is much more powerful than pre-Windows PCs, so why pretend it is something where C and Assembly are the only answer, when MS-DOS had plenty of languages to chose from.

Ok that's an argument for other languages, but not for JavaScript. JS is not good.

Just out of curiousity, how is Pike[0] for this type of use-case?

[0] https://pike.lysator.liu.se/development/git/

Look up how many Js basics tutorials there are, then check how many Pike basics tutorials there are

Prototyping and hobby use are the two obvious use cases I can think of.

I can imagine this would be great if you want to have multiple tiny apps running on a small embedded device - even with some basic multi-tasking! This is actually super cool.

Literally just yesterday I was trying to find the equivalent for Java, byte code or otherwise. I want to attempt to run several nano JVM apps on an ESP32, but I'm starting to think that it might not be possible.

Many years ago now I ran a custom 'script runner' that fit into 512 bytes of assembly language for a custom kernel (that was also 512 bytes) [1]. I would really like to play with something like Elk and see what can be done!

[1] https://bitbucket.org/danielbarry/saxoperatingsystem/src/mas...

It seems to me that embedded Lua would be a more appropriate choice, and barring that, there is already a quite robust embedded Python implementation.

Don't get me wrong, this is a cool project, but I don't see why you would ever choose to use JS in an embedded context except for fun. The arguments for using lua or micropython are already very narrowly applicable.

Seems inarguable to me that JavaScript is much more widely known and adopted than Lua. So it makes sense to use JS in 'toy' like environments where someone is learning an embedded platform.

JS is more common, but this is not a complete implementation. So those skills may not transfer 100%.

But they certainly will more than they would to a completely new language…

I don’t understand why people are so focused on preventing anyone from ever having to learn a new language. If you already know JS, learning lua should be trivial.

Interesting, could you link me the Lua and Python libraries you are thinking of?

Micropython and eLua come to mind

Microsoft also considers JavaScript makes business sense.

https://www.microsoft.com/en-us/makecode

Did you look hard enough?

https://www.microej.com/product/sdk/

https://developer.microej.com/supported-embedded-runtime-arc...

It doesn't take away from the point that the thing exists, but the licensing for this sort of stuff is almost always horrid. You're sparing yourself the headache of semi-supported open-source options, but it always feels like anything you saved by doing that is completely offset and then some by getting your balls busted in perpetuity by whatever vendor you went with. I don't blame anybody for sending these types of things to the mental trash bin when they see them.

Some supermarkets take pull requests, unfortunely most of them still like to get cash instead.

I'm gonna be honest -- I don't entirely understand what you're trying to say. Are you implying that the work needs to be paid for it get done? Of course that's the case, and open source embedded libraries can and do make plenty of money through consulting without having to annoy the customer with per-architecture and/or per-product licensing, and without having to hire salespeople to convince customers that that's necessary.

> Did you look hard enough?

I did see microej but I'm specifically interested in open-source - and from what I can tell this is some kind of paid service. I didn't look particularly close though [1]. The ESP32 device they do support appears to be a very specific development kit [2].

[1] https://github.com/MicroEJ

[2] https://github.com/MicroEJ/Platform-Espressif-ESP-WROVER-KIT...

Well you didn't mentioned you were looking after free beer.

There were several very small implementation of JVM over the years. One semi-popular one was leJOS for Legos Mindstorms. At one point, I recall even building it for DOS to test some things.

https://lejos.sourceforge.io/

Seems interesting, how was the porting process?

I was just looking at the Java bytecode and thinking "in theory, one could build a small JVM for this" [1]. But it still looks like an enormous effort to build, test and then use in a meaningful way.

[1] https://en.wikipedia.org/wiki/Java_bytecode_instruction_list...

IIRC, at the time, building on DOS w/djgpp was fully supported since leJOS is designed to be fairly OS independent. There's not much room for an OS in the Lego Mindstorms MCU. You can think of it a bit like how people today build minimal Rust or Go-lang OS experiments.

> Anything that good hackers would find interesting.

Totally on topic. I dislike JS but this is rad. And like upthread said, clean understandable implementation. Very educational.

Because "20KB on flash/disk, about 100 bytes RAM" is smaller than practically anything out there in the space.

Forth is at that size. Tcl implementations are at least triple that. As are all the Schemes I have seen. Lua similarly.

Anything I've missed?

Because your system may require that you can change code at runtime without reflashing your micro. I very, very much dislike JavaScript, but it's not an outlandish requirement and there's not a lot of (what I would consider) great options in that arena with expansive C interop (which you would need for a "real" application). There's basically JavaScript, Python, and Lua. There's plenty of other stuff out there, but none with the same amount of backing as those three.

Embedded development is the bat country of software. You will definitely have to do some unsavory things to get where you're trying to go.

Wouldn't there be better options for that? python, lua, tcl, perl, ... ?

ELK is also used to refer to the logging stack of ElasticSearch, Logstash, Kibana. Just FYI

It also refers to a species of deer.

Oh, and https://www.eclipse.org/elk, the Eclipse Layout Kernel

Let's not forget the Benevolent and Protective Order of Elks.

Pretty big oversight of the author not to call out their lack of association with the 153 year old fraternal order.

also https://en.wikipedia.org/wiki/Extension_Language_Kit

Why JavaScript? Why not some other DSL, and if you need you can convert a subset of ES6 to that DSL?

JS has some features that wouldn't make it ideal (e.g. no difference between ints and floats). I don't think it would be hard for developers to learn slightly different syntax.

If it's a big subset or the full language, I can see the benefits of using existing JS apps. But with a small subset and presumably for a small computer, I don't think most of those apps would work anyways.

JS certainly isn't ideal for µC, but it can work as shown here. A problem in the embedded world is finding developers as it is, maybe this could help?

I believe Arduino has a C++ transpiler. I would almost prefer Javascript here, although you could just use C as a subset.

Arduino has a C++ compiler, whatever the scripts are called, they are a plain C++ library.

Plenty of modern µC, e.g. ESP32, are more powerful than computers like these ones,

https://en.wikipedia.org/wiki/PC1512

As exercise for the reader check the programming languages available up to MS-DOS 6.22.

It is about time to move on from the mindset that µC are only PICs with 4 KB.

It has never been a question of computing power or memory, that was already solved decades ago. It is a question of fitting the µC to the application. Price and power consumption are usually the qestions to be answered. 0,50$ or 2$ chip still makes a difference.

I think ESPXXXs usually have < 500kb ram. Still quite tight.

The ESP32 has more memory than the PC linked above.

COM executables were 64KB, and EXE could use as much as they wanted from 512 - 640 KB in multiples of 64 KB, minus the MS-DOS resident size.

Naturally stuff like HMA came later into play with MS-DOS 5, which wasn't something that MS-DOS 3.3, again from the PC above, was capable of.

Or if you prefer, I refer to what was possible with 64 - 128 KB on Timex 2068, Spectrum 128 +3A (with CP/M), Commodore 64 with GeOS,...

Which, yes games would be coded in Assembly, there was business stuff being sold and coded in BASIC, Pascal, Forth,...

Why? Because the author wanted to make this project using JS & likely had a specific use case in mind. Plus there are many people comfortable with the language.

Why not some other? You're free to build one yourself for whatever language you choose.

Reply on news.ycombinator.com