A virtual machine for microcontrollers
blog.toit.io
blog.toit.io
Embedded development is tricky, and anything that can help improve the productivity is useful.
The problem is that there have been many, many attempts at doing this exact same thing over the years. And they all have to fight the same fight: to get some serious traction for their particular approach.
Back in the 00's, Sentilla did a Java VM for the MSP430. This got a lot of traction - they even had James Gosling himself vouch for the project.
More recently, Electric Imp and Particle.io have taken similar approaches, but with other languages.
Micropython and a variety of mini-JS engines are also around. Espruino, Tessel.io, and many more.
In a technical sense, running a virtual machine on a microcontroller is a really good way to go. You remove a lot of low-level friction that you otherwise have to deal with. And you get a lot of things almost for free, such as Over-The-Air (OTA) updates, and more control over security.
But when you get down to actually doing the work, you frequently end up wanting to have all that low-level access, even if you have to endure a bit of pain. Sometimes you don't even want to have an operating system with a hardware abstraction layer. You kinda want the bare metal.
So Toit are in a tricky situation. They may have a really good product. But they will have to get some serious traction. Developing a rock solid programming language and virtual machine is hard. Really hard. But it is way easier than to get that sweet, sweet traction.
Then every time there's advancements in what you have available, people add all the niceties back. And at this point it is not embedded anymore. It's just a low power full device. Just like java for the MSP430 (like anyone would spend $7.99 per chip for a TV remote with 2week batteries, instead of $0.004 and 2+month)
True embedded development usually doesn't even have room for a watchdog timer. let alone virtualization and memory protection.
This is not new. This is not embedded. This is just the usual cycle of hardware generation changing and people in academia ivory tower (or worse, google solid gold tower) having nicer toys than everyone else.
An ESP32 (and oh man they’re great) is more powerful than a mid-late 90s desktop, but power isn’t what separates a computer from a microcontroller. It’s the abstraction compared to the bare metal control.
(To be clear, the project in the post is very much embedded, but programs running on a vm written in a high level interpreted(ish) language are not so much.)
embedded is a term that means your code is closer to the discrete component analogy as possible, with all the same design and testing expected from the electrical design.
If you want to add modern language niceties to it, fine, But I would say that anything that brings in the "app" concept, and specially installation and fleet management, the better term is microcontrollers.
And there are perfectly fine open source solutions with the same goals, such as https://archive.fosdem.org/2020/schedule/event/ioterlang/
Maybe the industry would benefit from some more fine grained terminology here, perhaps call the small stuff MCU embedded?
But that's not the important part. I think one major problem is that the term "embedded" puts a lot of engineers into a frame of mind that rejects programming in anything high-level. We have done lots of projects where something like MicroPython would have been perfect, but every time I try to sell something like that, I get a bunch of excuses that really aren't grounded in any concrete objection and we go right back to twiddling bits in C.
A watchdog timer is often a piece of functionality internal to an MCU. https://microchipdeveloper.com/8bit:wdt The majority of embedded development does use watchdog timers, at many levels.
Java is running inside of this card, https://zdcard2.en.alibaba.com/product/60655706921-801652433...
If a given block of WASM isn't performant enough in the interpreter, then drop it to compiled code and expose it to the rest of the WASM - at least then only a small fraction of your code requires a painful firmware upgrade, the rest can be updated in a user defined section of ROM. It would be great to have a compiler/linker combo that could use annotations to configure which parts of the firmware should be compiled and which should be interpreted. With inlining or even just AST substitution (replace global vars with register access), it should provide all of the low level access necessary while still allowing for high level Arduiono style libraries
A modified WASM with explicit jumps and less “magic” behavior would be great. Perhaps a transpiler could generate such a “interpreter-friendly” pseudo-WASM to be loaded on the microcontroller.
(FWIW, even though WASM supports static data blocks, it also has provisions for dynamic allocation, and most WASM programs over-allocate their static blocks to make room for a C stack. So, in practice, making WASM work for microcontrollers and static data would also mean new languages or compilers that are much more conservative with memory use.)
Articles like this hide this complexity by talking about "microcontrollers", but behind that innocent facade hide utterly different beasts like the Atmega (of Arduino fame), MSP430, PIC, SH4, and anything Cortex-M3-based, just to cite a few.
It's not just about the generic memory and compute constraints of the genre as a whole, but adapting to the specifics of each platform. Solved the platform/tooling for the ESP32? Great, but that doesn't help my STM32-based project.
To make a crude analogy: the familiar "PC" architectures are pretty consolidated (x86, x86-64, or arm32(Thumb), arm64), but they're like the primate clade compared to the microcontroller "animals" that range from the carpenter bee through the sponge.
The server end of their platform provides the management and app deployment infrastructure. It's claimed that the RTOS and VM environment makes deployment reliable even in the presence of failing apps. And that one app on a IOT device will not prevent other apps from running compromise management or brick the device.
Toit.io, the company makes $$ by charging you .10/MB after the first 100MB/mo for use of their management and app deployment and you are left free to choose your own data platform. What I don't have feel for is what sort of utilization per device I can expect for the management and instrumentation traffic.
*Edit I dug around the website and the tooling including the language looks like it is mostly closed source.
However, I'd be very, very hesitant to develop anything significant on this platform. At the very least I'd want to have a. the tooling to run unit tests / simulations of my programs on the PC before deploying and b. a clear migration path for if they went bust.
Their value add is the infrastructure around deployment, so why is their language closed source?
I really like the Balena.io approach - the whole platform is open, so if they do go bust you could run it yourself, but they provide real value in running the infrastructure - so it makes loads of sense to pay them.
Depending on what "management" entails, that is. It might only mean that no more updates are possible!
> In a nutshell, the problem is that on microcontrollers everything is firmware that is compiled, linked, and deployed together using really old-fashioned tools. Changing anything means changing everything.
The author hasn't seemingly heard of http://www.ulisp.com/ which is small enough to fit within kilobytes of memory and at the same time featureful enough to drive complex microcontroller applications. At the same time, it has a normal REPL running on the microcontroller that is accessible from the outside and usable for Lisp-style interactive programming.
> If one wants to do serious stuff they would use an appropriate RTOS and program it in C.
It's unfortunate, but still largely appears to be the case. I find C very time consuming to program, so I ported Nim to FreeRTOS [1]. It's _very_ nice being able to go from writing highly optimized ISR functions to high level JSON parsing in one language. Add in defaulting to memory safety but with no pause-the-world GC. I tried Rust but it seems more difficult to integrate into existing world RTOS'es, flashers, Swagger debuggers, etc.
Though, I've been curious what running a WASM VM would be like? One could integrate any language: C++, C, Nim, Rust, etc. Would be interesting.
> MongooseOS does more than this if we're talking ESP32, also other devices, Javascript, C, C++, commercial support, cloud based OTA upgrades and integration with AWS, Azure, Google and IBM Watson IoT cloud services.
MongooseOS does seem interesting, but very targeting a niche market with prebuilt needs? For future RTOS'es I think ZephyrOS [2] has a lot of potential given it's now supported by NXP [3], TI, and others but is independent of any given (cloud) vendors or other IoT companies. Some might not like the CMake based build system, but in my view all the RTOS build systems are terrible in their own special way.
1: https://github.com/elcritch/nesper 2: https://www.zephyrproject.org/ 3: https://www.nxp.com/design/software/embedded-software/zephyr...
Zephyr is nice and well documented but support for ESP devices is quite lacking. For NXP devices it's great.
From Fabrice Bellard himself:
> QuickJS should be able to run on the ESP32 platform as it is OS independent (as you said, quickjs-libc.c is not part of the engine). For simple scripts it should fit in the available RAM.
https://www.freelists.org/post/quickjs-devel/quickjs-on-esp3...
---
Some attempts on GitHub:
https://github.com/binzume/esp32quickjs
It might make adoption harder but I'm glad they did because it looks like a really nice language.
I don’t know ulisp. I believe with MicroPython you basically have one monolithic Python application running in the interpreter on the device. I believe their claim is that their system enables you to have several applications run in their interpreter with reasonable isolation between them. So I think it’s more about having a single monolithic application vs a set of specialized applications / services that you can combine.
I believe their original system OOVM did allow for lisp-style (smalltalk-style) interactive development. I don’t know if their new system supports it too.
Is there something similar for other language environments?
There are ways to tether a microcontroller to a host IDE, like Linx for LabVIEW or the firmata protocol for lots of languages, but those methods don't let you go from running code on the workstation to running code on the target (easily) like I think you're describing. http://firmata.org/wiki/Main_Page
The Mathworks' Matlab and Simulink arduino stuff advertises some capability in that regard but I haven't seen it myself.
However, with the amount of RAM/FLASH on modern MCU's you can implement the "compiler" code in Forth. I used a similar scheme for a while during experimental phases, but have moved away as Forth code becomes tedious to modify after you haven't looked at it for a while (e.g. a few weeks). It's just easier to setup OTA firmware updates.
Modern MCUs have so much RAM and flash you could probably run a whole 1980s style development enviroment on them (think say, Turbo Pascal and a collection of tools) on a console, TUI and all.
— Dan Ingalls
Found the quote in Byte Magazine Volume 06 Number 08 (Smalltalk) page 298
https://archive.org/details/byte-magazine-1981-08/page/n311/...
So what insight is this quote giving us today, does it still say something interesting, and is it still true in certain ways?
I don’t have a Smalltalk history or know much about projects like LispOS or Toit at all, but I do have a clear (to me) picture of what an OS does that is useful and should not be part of a programming language. One example of that is working on console games, for example Nintendo consoles, before they provided an OS. It was a nightmare because the game developer was on the hook for handling a very long list of abnormal system conditions. The certification process for a game on a Nintendo console required all developers to conform to standards that specified exactly what errors needed to be displayed under what circumstances. Game developers needed to handle cases where a player would accidentally bump open the CD tray or pull out a cartridge in the middle of a level load, or repeatedly yank and replace a controller cable, things like that. Think about how silly it is that every single game developer, thousands of them, had to - separately and individually - spend tons of time re-engineering the same solutions to these things that the console itself should have handled, things that a couple of people at Nintendo could have engineered once for everyone. Well, now they have an OS, and this is only one of the many reasons why. It might not be immediately apparent that Nintendo’s certification standards have any bearing or say on what a home computer should be like, but UX standards in system-wide error messages are important, designated responsibility for which process handles hardware errors and user notification are important, and making sure that efforts that have to be common to all programs are implemented in such a way that devs can’t mess them up and don’t need to reinvent the wheel are important.
Note: Ingall's quote was about the programming language in general, and in the context of home (desktop) computers. Things are different, and my reply here does not apply, if we're talking about microcontrollers like in the Toit article.
> There is no reason we can’t get back to and surpass this previous state.
Yes there is. There are a whole bunch of extremely critical reasons to have an OS and to separate it from a language, which is why we have them and why we’ve always had them (on desktop machines). I already gave some reasons above, but the reasons in my Nintendo story are some of the least important reasons there are today, and they’re still pretty important.
Here are a few other reasons:
Security. Some processes should be allowed to have special access and do things that most processes cannot. Think about what you’re suggesting if you remove the OS/user boundary: it means that daemon processes written by other people have root access to your system. You do not want that no matter what you claim to want in a programming language.
Management of shared resource contention is something an OS should handle. Do you really want to have to write your program to play nice with the network, hard drive, and GPU? I don’t, it would automatically add months or years to any development projects, even if you had libraries and language features to support it, because it would force you into an asynchronous programming model with a responsibility to handle a large number of error conditions (most of which are out of your control).
The OS handles virtual memory paging, so you would be on your own for providing a memory system that can have a resident size greater than available RAM. Not only that, every process would be on their own, there would be no shared paging file. (When you think about the paging file, don’t forget security).
Other simpler reasons including program bootstrapping (loading and execution), shell & file navigator access, shared system settings (display, audio, network, etc.), temp file creation, etc., etc., etc.
The difference between embedded devices and desktop machines is another good reason not to bake the job of the desktop OS into a programming language. So is the fact that there is more than one programming language - even at it's simplest, the OS boundary makes a great language agnostic interface. (Why should every language implement it's own storage, networking, and display? Wouldn't that be a complete waste?)
I can’t think of any good reasons why there should be no OS/user boundary, so that is my question: why do you want that? What good would it do? How would you handle virtual memory, shared resource contention, and security, if there was no OS/user boundary?
I have to admit it’s interesting in the context of virtualization, where deploying a program to a unikernel virtual machine might be perfectly fine for a lot of programs. In that case, some host OS is still handling security and resource contention, so this seems a little like ducking the question.
The unikernel design in practice does not put the kernel into the programming language either, it just allows compiling the kernel and the language together. It still has an OS/user boundary. Security is either non-existent or very difficult with unikernels. Running multiple programs at once is tricky.
“unikernels are unsuitable for the kind of general purpose, multi-user computing that traditional operating systems are used for. Adding additional functionality or altering a compiled unikernel is generally not possible and instead the approach is to compile and deploy a new unikernel with the desired changes.”
Maybe, a bit, if it's a chatty OS, but, mostly, no, that’s a shell.
**** COMMODORE 64 BASIC V2 ****
64 K RAM SYSTEM 38911 BASIC BYTES FREE
READY.The microcontrollers you would use this on (i.e. not the absolute bottom end, which would be too slow for a VM) may not have a MMU, but they do have a MPU. That's enough to get process isolation. And implementing relocation to load an ELF image isn't rocket science.
Also, I'd like to point out the billions of embedded systems implementing safety critical functions like huge industrial robot controls, or the brakes in your car. They're all built (and certified) without a VM. As a matter of fact, the VM would probably make them fail certification, unless it is specified and verified itself to a very fine degree.
I realize that there is a pattern: when high-level programmers talk about security and reliability, they basically mean "hackers breaking your system to steal your passwords". To the point that memory bugs = security vulnerabilities, and nothing else.
This, of course, has another meaning for embedded.
> there's no point to distinguish them
I think the distinction should be clearly made when someone wants to sell me something like Rust. They come up with the daily link about 70% of the bugs are memory issues plus something in the lines of "memory vulnerabilities! think about the hackers! exploits!", when it's clear that these arguments don't click in (many areas of) embedded. Safety and security have another meaning. The steal of a password is the least of my fears, if I think a bad implementation of mine can chop-off the hand of an operator.
(while the terms are intermixed in general discourse, both the embedded and security worlds consistently separate the two, at least in everything I've seen)
And VM == bye bye JTAG debugging, which makes it DOA for most developers past the Arduino level.
Yet don't put too much trust into them.
https://hackaday.com/2016/10/24/toyotas-code-didnt-meet-stan...
That said, the really bad safety issues in a car are probably on a much higher semantic level, i.e. "at X level of braking, do Y", which neither a VM nor a safer language like Rust could catch.
https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_... https://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRU...
I'm sure that if you want to avoid
uint8_t* ptr = 0x20000000;
while(1) { *ptr = 0xFF; ptr++; }
But this is a very interesting project in the way it's presented. I did once something similar, running in parallel some sort of reduced LUA scripts.
Even though, after 20 years of working in embedded, my free and unsolicited advice would be this: if you want to learn embedded, no matter how hard you are trying to avoid it, you need to learn some low-level language like C (or Rust or whatever is fancy now).
Put your mind at ease and do the extra mile. At the end learning C is not that hard. And I suppose you are learning, because if you propose Toit in your workplace is because you don't know how to code anything better at the moment.
I don't want to hurt anyone's feelings or sound dismissive but... learn C (or anything low-level). The rest are just toys of the moment.
On the other hand, C isn't hard to learn, since the language is so small. What's harder is to learn its pitfalls and the parts that are actually implementation-specific, and not specified.
I'd argue that learning some assembly (enough for a few toy projects) is more useful to understand the system. C is required if you want to make the most out of the hardware, at least for now.
I knew that this would be mentioned but I didn't want to write an extensive comment.
In many embedded markets, cents make the difference. Also, space constraints, power consumption, availability, etc., makes you reconsider overkillability/price relationship.
Someone will put a 16KB MSP430 in the BOM and there you go.
What I wouldn't buy is something like "hey, chips are cheaper, let use a prototyped javascript-system over a VM over an RTOS, just because".
> What's harder is to learn its pitfalls and the parts that are actually implementation-specific
Yes. There are tradeoffs, like in everything in embedded.
> It's probably faster to prototype something with micropython than C.
Personally I wouldn't know where to start with micropython.
Most of the stuff people do in the Makers community is more than doable with Micro/CircuitPython + a bit of Assembly.
You earlier made an appeal to authority that people should just learn C/low-level language, and then admitted you have no clue how you'd get started with micropython. What makes your appeal to authority... authoritative, then?
Not saying your conclusion is wrong, but here's some unsolicited advice, if you may take it: maybe you could explore the approaches you dismiss before professing that they shall be dismissed.
I do know that if I want to learn it, it isn’t something out of my reach. But there are things you can do with C you can’t do with micropython, so other than from academic standpoint, what’s the point? I am already proeficient with C.
That said, I don’t care what other people do. I had good intentions with my advice, talking from 20 years of experience in the field.
You can define structs, loops and functions on the fly and execute them. You can build a complete driver in interactive session, first by poking around in registers and seeing that the HW reacts per spec, and then assembling correct operations into initialization function and updating function. All of this is throwaway code but it can save large amount of time.
Maybe non-critical systems could be left with the micropython implementation, but so far I haven't learned how to profile and optimize it to satisfaction.
Oh, me neither, and I wasn't really talking of big production runs. If you have tight margins, you better take anything you can, and C is probably one of the first tools at your disposal.
> What I wouldn't buy is something like "hey, chips are cheaper, let use a prototyped javascript-system over a VM over an RTOS, just because".
You'd be surprised. The Harmony remotes comes to mind as an (old) example. In environments such as startups, time to market sometimes trumps even common sense. And people (including you and me) just prefer the tools they know most.
> Personally I wouldn't know where to start with micropython.
Funnily, I have never used it (except for a bit on a numworks calculator), but I can't imagine it being difficult once it's up and running on your microcontroller of choice. You probably flash and run as usual, except it's python code and has a repl.
For example I would rather use one of MikroElektronika Basic or Pascal compilers.
If you're interested in hearing Kasper talk about toit, he did a long livestream the other day, the toit bit is at the 1hr 34min mark. https://youtu.be/k7YITNpvcaY?t=5640
Not necessarily. There are modern and pleasant workflows using Rust.
This article is somewhat general. Ie, what is his intended use case? What is he building? Part of the beauty of embedded programming is you can avoid the complications of abstractions like virtual machines and operating systems, and the performance penalties that accompany them.
Now most projects will use C due to space issues, but that’s a different argument.
Those companies manage to stay in business selling such compilers for at least three decades now, which kind of proves there is a market of people willing to pay for them, instead of going with C like everyone else.
If you're using off-the-shelf consumer-y ESP things, you're likely insensitive to cost and you're probably paying for performance you don't need.
I last saw this technique (maybe 15 years ago) used by a super tiny .NET runtime environment intended for devices like smart watches; you wrote to "tiny" APIs, a tool did a bunch of reprocessing of your binary into smaller tables and whatnot, and poof, it would run on a watch. With all the GC and so forth going on, it's not something I'd want to write a whole embedded system in, though it was okay for applets.
I'm not oblivious to the security advantages of using a bytecode interpreter, but on a high-volume product you'd have to make a case for how important this kind of thing is.
What gets me is the implicit promotion of javascript as the gold standard of available tooling. That gives me hives, even though it's probably true.
https://github.com/Moddable-OpenSource/moddable#moddable-sdk
Why would someone choose Toit over Espruino or QuickJS, which seem to promise JS on a microcontroller?
The company sales pitch seems to be about fleet management and over-the-air updates. This doesn’t seem very relevant to me considering that I’m a hobbyist flashing a single microcontroller over USB, and I never run more than one program at a time. But perhaps I’m just not the target market and I’m not imagining the right use cases?
For me, a solid, well documented HAL that works well is key to efficiency in embedded development.
Rusts language features and the ability to abstract things like SPI or I2C peripherals that were once impossible in C are totally doable in Rust, and the community is growing.
Highly recommend checking it out
How so?
The `embedded-hal` (https://github.com/rust-embedded/embedded-hal) are these abstractions that allow this to happen
Has an emulated OLED 1306 display.
Kasper Lund has a lot of my respect for work he did on V8 and Dart so I was hoping this was going to be super interesting article about trying to struggle to.... make V8 run on a specific microcontroller!
I think I feel more cheated because of his background and that it's highly featured on HNews.
"It is fast, but prohibitively expensive in terms of memory."
So there's no room for v8 on a $2 board, Toit vm and toit language have been designed from the ground up for the iot world.
Tidbit, Dart team did have an iot project called dartino, before flutter took over the dart world.
Anyway the article was posted on the toit website.
I've been following their progress via twitter. https://twitter.com/toitware https://twitter.com/toitlang
Yeah, as someone who works with microcontrollers professionally and would always love better tools to work with, this article didn't do much to sell me.
It is a nice bit of marketing in the way it positioned itself. I don't think I'd be able to articulate why I'm not rushing to use this without getting called an old fogey.
If successful, the streaming service with lack of relocation should be a useful tool in the toolkit for working with microcontrollers. I don't see much use for it for me at the moment, but then I haven't used uLisp much either.