ResinOS, run Docker containers on embedded devices
resinos.io
resinos.io
Has your thing got always-on sensors, or actuators, or both? Your job title is probably "embedded software developer."
A specialized or dedicated computer used to control devices (such as automobiles, home and office appliances, handheld units of all kinds, etc.) where the operating system and application functions are often combined in the same program. An embedded system implies a fixed set of functions programmed into a non-volatile memory (ROM, flash memory, etc.) in contrast to a general-purpose computing machine. However, sometimes single board computers and rack-mounted computers are called “embedded computers” if used to control a terminal interface, machine, motor, etc. An embedded device or system may contain more than one operating system and/or processors (microprocessor, microcontroller, etc.). Specifically excluded from this definition are all types of enterprise computing machines deployed as general-purpose computers (i.e., desktop PCs, standard laptop PCs, enterprise servers, etc.). Our definition of embedded system/device is intended to give a good indication of the potential operating system and run-time software royalty opportunity within the embedded systems market.
So there you go. As you can see our definition has grown substantially over the years as the embedded and IT/enterprise markets have converged. It's much more helpful to think in terms of vendors though.
Traditional embedded silicon vendors include NXP/Freescale, Renesas, Microchip, Rockchip, Cypress/Broadcom etc. Traditional embedded OS vendors include Wind River (Intel), Green Hills, Express Logic, Enea, Micrium, etc.
Intel/Samsung/TI and Microsoft attribute small portions of their revenue to embedded systems, but they don't show up often in conversations with embedded vendors in their respective markets...they focus on enterprise servers/computers/software so they are usually less relevant to the day-to-day going ons of embedded vendors (until they swoop in with the intent to acquire).
This is changing though...has been fun advising some larger players on embedded strategies. Interesting time to be in the market for sure.
Um, what? Pretty much every chip TI makes goes into either an embedded or softwareless system. Analog, mixed-signal, and DSP have been their bread and butter for decades.
From the developers view these Linux based systems are pretty much general purpose computers.
However what mostly counts for the definition of "embedded system" is the end users view. The end user (customer) gets a complete system for a specific purpose, he can't install any own applications and partly can't even update the system.
I always loved Arduino as it promoted a few core ideas (apps are highly portable, the environment is standardized over boards and devices) - this allows the hobbyist / stack overflow reader to make some awesome hacks in a few hours and really reduces the barrier to entry for maker projects. However, in the last few years, I've wanted a similar type of environment for all the Linux boards that now clutter my desk - the ability to run different apps on multiple boards without having to understand the depths of GPIO alternative function mapping and quirks of various distros.
Having a docker container approach where the app is sandboxed away is the right approach I think for this, but there should be a "standard API" as well put in place that the apps that be written against (and bindings to python, rust etc...) that is also emulated in a desktop or webpage environment. This container doesn't support any HDMI output or full Debian environment - just a basic API to start with. Add PIP/Cargo etc.. for advanced use cases. A movement away from Arduino might be around the corner if there is something as accessible but much more powerful available. The block is no doubt "power management" - suspending/hibernating Linux boards for battery operation etc...
I hacked up a version of resinOS last year (using Android as the base Linux port) and running docker on a root FS but never took it anywhere as the day job (a different kind of startup) takes most cycles still. Can't wait to install this later on a few Pi's :)
What are RasPi and similar boards lacking? Power management? Not enough GPIO pins?
For me its the ability to "blink" (sleep a designated time and then re-awaken for a brief burst of activity) and so perform some task for 6 months straight on a pair of alkaline AA batteries.
mark@helium.com
[1] helium.com [2] https://store.helium.com/ [3] https://www.helium.com/helium-os/
It has minimal dependencies and doesn't even need a virtual-env. It's prepackaged in rpms, debs and apks. It needs raw access to GPIO and USB. ResinOS, which technically interesting, seems like way too much and another layer I just don't need.
I don't want to sound like a dick, because I'm sure the authors put a lot of work into this, but I really don't think this is a good pattern at all.
If I had to write my client again today, I'd probably write it in RUST/LLVM, giving me even less of a reason to stick it in a docker container on my device with <8MB storage space.
The point of containers is to make updateability easy. To bring embedded software closer in line with what's going on in the cloud. To enable the kinds of workflows we have had in every other part of the development world for decades. It may or may not be right for your project. So long as you understand the intention and make informed choices about the tradeoffs, there will always be cases that fall in the one or the other side of the fence. I don't perceive this as being a dick, you're stating a reasonable case for a model that does indeed work for a lot of cases.
We will continue working to reduce overhead, hopefully power requirements and cost will go down, and somewhere along the curve, we may even meet your needs at some point. Whatever the case, we're pretty sure we'll never cover everything so if we gave the impression that resinOS will end all embedded OSes, then it was the wrong impression to give. :)
Can you give some examples of types of software that would benefit from this?
We don't deploy fast. We deploy once.
You can push out regular updates, but you have to be very careful. You need to have full testing environments and real lab units of all generations to test on before you push to production and realize you now have a brick on the roof of an 80 story building that takes a week to get a roof permit form.
The cloud stuff is gibberish. Today's cloud is rebranded hosted platforms. I agree, I don't see the argument of making _embedded like the cloud_ being meaningful either. I know on my embedded projects we had a tight ship pipeline and you need that for the reason stated above. Sure you can do that with Docker/ResinOS, and it might help with building a piece of that, but I have a feeling it adds as much complexity as it might mitigate.
Like the discussion at the top of this thread, we've really muddied up the waters when it comes to defining "embedded devices".
There are devices like mine, which are the early definition of the term (single purpose, low cost, tightly defined I/O) and then we have low cost highly compact computers, literally servers, that can be multi-purpose and are internet connected 100% of the time by design.
No.
The difficulty in performing upgrades is not in upgrading the code.... that's relatively easy.
The real difficulty is making your data and the current state of your application upgradable.
This is not going to be solved by containers.
Do you have anyone using Kubernetes on ResinOS by chance? I've got a small fleet of small hardware and had started looking at running k8s on HypriotOS, but this looks a bit more custom built.
Thanks for the heads up!
Familiarity with Docker will allow makers to get multiple runtimes up and running on higher-end (but small) SoCs...but for what type of applications?
(Disclaimer, we've been doing exactly that since 2011. I've been wanting to move to a containerized approach for years, but our customers are happy enough with the current solution that they don't want to fund it. Alas.)
Is it possible or has any thought been put into making it possible to use Yocto for building the containerized images that are deployed to resin (probably in the form of docker images)? Although this is less convenient than using off-the-shelf docker images and customizing with your own docker files, I believe many of the benefits of Yocto in terms of building a custom embedded linux system might also be recognized within the context of resin.
What does support of the other processors entail? Is it a kernel modification? Are you sure the dsp on the Beagle doesn't work? Say hi on the gitter channel ( https://gitter.im/resin-os/chat ) and the resinOS team will be there tomorrow morning (european time) to talk more.
Also curious about the Raspberry Pi story.
ResinOS does indeed have a smaller footprint, a broader set of supported architectures (Snappy only supports ARMv7 and above), and uses Docker instead of LXD+snaps.
RancherOS has a built in Go PID1 that starts a system Docker for the system services, and a user Docker that you then run your apps in.
This also means your console, and user container run time is a switchable container image.
Yep, RancherOS works on ARM, and we're building out our build and test infra.
Mostly, we're all working, cribbing off each other to get to our individual goals.
Just skimmed a little bit through the resin-os docs and was left with the following questions:
- What's your plans for audio and video? The base system doesn't seem to provide any audio or video manager. Should those (e.g. wayland) also be ran inside a container, which could make access from other containers difficult? Or should they if required be included through the Yocto image build process? Or is A/V currently out of scope for you and you focus on pure networked devices?
- Haven't found any description on how prebuilt docker images can be packaged and flashed during the installation process or later on? I only found the rdt push description which seems to push the application sources to the device and builds the image there. This might be interesting for development but would not be anything that I want at all for production. E.g. I don't want to give customers my app sources, I need company git repository access (credentials) during image building, etc.
- What's your reasoning about building all those build and development tools in coffeescript for node? From a pure tool user perspective I know that it's a turn-off for lots of users that it requires node6 on the PC (it's not LTS, other stuff might require other versions, what is that node-thing anyway, ...). And from a developer perspective I would rather not prefer an exotic programming without static typing and a rather volatile ecosystem for a reliable long-term system. I don't want to say here that it isn't possible to create solid project with it and yours might work great - it's just not the first option that comes to my mind.
Preloaded containers is something we have built out for resin.io, and will be extending to resinOS standalone as well. The current release is just the development version. Production version with preloading should follow.
We use node.js for the development toolkit, not for the OS components. There will always be programming language disagreements, but we find we are productive in node.js and it works for us. Coffeescript is admittedly getting a bit long in the tooth and we should switch to ES6 or TypeScript, but I personaly see that as a minor change, though a good one.
The number of containers is kinda a red herring for a couple reasons. For one, you can make a trivial container and launch as many containers as you like, it really depends on what's in them. For another, embedded devices don't usually need hundreds of containers, they run payloads that are either single-container, or look a lot more like a docker-compose file with a few microservices.
ARM isn't anywhere near as standardized. There are usually vendor kernel patches, not just for drivers, but dependent on the block layout of that particular ARM board. uBoot is often forked and also specialized. There is no one shoe fits all Grub.
I'm really interested in the answer also!
Why does it take so long to boot? Is it limited by the speed of the SD card? I wonder how it would do on a Zynq platform with QSPI flash.