Hacker News Reader

Comment by mgsouth | Hacker News Reader

Waaaaayyyy back in the day (1988) it started out as "Real-Time Executive for Missile Systems" [0]. Hard-core hard-real-time.

Today the "MS" stands for "Multiprocessor Systems". Heterogenous multiple processors [1]. The OS is perfectly happy running x86, ARM and RISC-V in the same system, though I don't know which of the canned Board Support Packages use that capability.

It's a pretty good introduction to RT--small enough to get your head around, but lots of capability. I used it for my first major embedded software project (LAN NIC driver, based on Linux driver for similar chip) 15 years ago and it was pretty straight-forward. Rate it 4 1/2 stars, would code again.

[0] https://devel.rtems.org/wiki/History/Timeline [1] https://docs.rtems.org/releases/rtemsdocs-4.8.0/share/rtems/...

View on HN

I have also used RTEMS for my first major embedded software project, almost 25 years ago.

I had to port it to PowerPC, because that was an architecture not yet supported at that time, but the porting was very easy, because the code quality and the documentation were excellent and all parts dependent on the CPU architecture were clearly separated from the rest. I also had to write some device drivers, including an Ethernet driver, and again, the documentation and the device driver examples made that task easy.

RTEMS proved to be a much better choice than the alternatives at that time, i.e. pSOS and VxWorks, because the access to the source code enabled a degree of customization impossible with the proprietary RTOSes and its performance was great too.

In later years, I have not used RTEMS much, but not because I have discovered any better RTOS, but because I have realized that for a majority of the embedded software using an RTOS is an overkill. It is simpler and with less overhead to just use on bare metal an event handler with a priority queue storing scheduled actions whose time duration is limited (this is equivalent with cooperative multitasking).

There are applications which can be simplified by the preemption offered by an RTOS, like RTEMS (typically when porting to an embedded device some programs that have been written for a general-purpose OS, e.g. for Linux or FreeBSD), but the need for this must be assessed carefully, because using preemption guarantees that you cannot use the CPU close to its full capacity.

Therefore, using RTEMS (or any RTOS, but RTEMS is one of the best) can be the best choice for an embedded system when the CPU is quite powerful in comparison with what it has to do, so optimizing the CPU utilization is not needed, and the main target is to minimize the software development time by reusing some programs which have not been written specifically for real-time systems (e.g. a Web server or other complex networking programs).

Without an RTOS, i.e. without preemption, it is mandatory to be able to estimate upper limits for the execution time of any function, to be able to guarantee that deadlines are met. With an RTOS, you must still be able to do that for functions that are executed by high-priority tasks, but this may be unnecessary for those executed in low-priority tasks (which will be preempted, whenever necessary, by the high-priority tasks).

I used RTEMS on Movidius Myriad vision CPU (running on LEON- a SPARC clone).

I remember the RTEMS event concept was not very good- the events are globally shared. You are better off using semaphores.

>I have realized that for a majority of the embedded software using an RTOS is an overkill

I agree with this. Instead of an OS, we like to use a small framework that provides a CLI, database and UART communications (YMODEM protocol). It's a single stack system that provides a simple work-queue for scheduling callbacks, and for interrupt code to schedule non-interrupt code. It works for almost everything as long as you are comfortable programming in an event-driven style.

https://github.com/nklabs/libnklabs

(recently added support for AVR 8-bit processors).

Holy cow, I really didn't read into the MPCI interface enough. Some of the capabilities mentioned in the docs are impressive regardless of system, let alone an embedded real time one...

I'd love to hear more about your experiences!

Not much to tell. I was looking at maybe using an RTOS to reduce VoIP jitter. Picked RTEMS on a generic x86 PC, there wasn't a driver for the NICs we were using, so wrote a driver using a FreeBSD driver (for an unrelated chip) as a base template and a Linux driver for a related chip as a kind of spec. Don't remember if had to do the top-end or just the bottom-end for the NIC. The driver worked fine on RTEMS, but the UDP stack was basically straight from FreeBSD and was definitely not real-time. Wound up not being much of an improvement, it wasn't worth rewriting the entire UDP stack or introducing RT to my non-RT team, so dropped the idea.

Did find a performance bug in the core I don't remember much about (unfortunate choice of operations that flushed caches or something and realllly slowed down... something). Submitted the half-a-dozen line patch upstream that Joel Sherrill pulled without drama, even though my test case was overly complicated and opaque.

What limited the networking stack in terms of RT? I'm having difficulty imagining where the real time blockers would be, but curious as thats a big motivator for me.

We wanted to handle 10's K of VoIP channels, on multiple processors, with each channel sending a UDP packet every 20 mSec +/- a couple of mSec. The network stack was just the standard FreeBSD "box" and had no provision for that kind of clocking or time-to-process guarentees. Our application could call send() on a fixed schedule, but couldn't control when the packet would actually pop out. The process time would vary because of things such as global locks for buffers, varying code paths (do we ARP?), and varying amounts of competition from other processes. The only way to get reasonably consistent times was to way over-provision the system so it was loafing. We were already doing that with Solaris, so not worth it.

What we needed was something like Linux's scheduling queue disciplines [0]; it would have allowed us to say "here's a packet, schedule it for 10 mSec from now", and we'd be good as long as the vast majority of packets took less than 10 msec to wend through the stack. But that didn't exist at that point.

[0] https://tsn.readthedocs.io/qdiscs.html

In my experience, RTEMS works great if your scheduling/timing is in the ms-range but if you need precise timing in the us-range, especially on smaller MCUs, then you will run into lots of trouble with jitter from different sources, for example because certain timers are checked sporadically, messing with your cache. I don't know if the situation has improved since then.

I believe that what you say is correct when you use RTEMS with one of the existing BSPs, with no changes.

Because RTEMS is open-source and also because its internal structure is many orders of magnitude easier to understand than that of something like Linux, or even of some *BSD, extreme customization can be done with relatively modest work.

When you do not need them, it is easy to remove completely the implementations of many parts of the API and of many classes of kernel objects, to obtain a much leaner RTOS kernel.

Any causes of jitter from the kernel or device drivers (if they occur in higher priority tasks, as the lower priority tasks should not be able to interfere) should be relatively easy to identify and patch.

I have used once RTEMS for an application with many channels of synchronous multiplexed telephony, with very tough jitter requirements, and there have been no problems, but it was not RTEMS out-of-the-box, it was customized for the application, i.e. with about half of the kernel removed, a new interrupt handler added, and with custom device drivers.

Reply on news.ycombinator.com