I understand it's less elegant, efficient, etc. But sometimes commodity wins over correctness.
For automotive and aerospace, you really want the control systems to be able to run no matter what.
I'm unclear on why you put "many cycles to spare for misses" in quotes, as if it's unimportant. If a linux/arm (or x86) solution is displacing a much lower speed "real real time" solution, that's the situation...the extra cycles mean you can tolerate some misses while still being as granular as what you're replacing. Not for every use case, but for many.
Maybe they used the quotes because they were quoting you, haha.
As for the quotes, it was a direct citation, not a way to dismiss what you said.
Actual realtime systems are about workload scheduling that takes into account processing deadlines. Hard realtime systems can make guarantees about processing latencies, and can preemptively kill or skip tasks if the result would arrive too late. But this is not something that the Linux kernel can provide, because it is a system property rather than about just the kernel: you can't provide any hard guarantees if you have no time bounds for your data processing workload. So any discussion about -rt in the context of the Linux kernel will always be about soft realtime only.
In the case of the kernel, it is indeed too large to be considered hard realtime. Best case we can make it into a firmer realtime than it currently is. But I would place it nowhere near avionics flight calculators (like fly-by-wire systems).
For example, if I have a video game where there's collision against the walls, I can understand this as potentially colliding against "multiple things simultaneously", since I'm likely to describe the scene as a composite of bounding boxes, polygons, etc.
But to get an answer for what to do in response when I contact a wall, I have to come up with an algorithm that tests all the relevant shapes or volumes.
The concurrency bug that appears when doing this in a naive way is that I test one, give an answer to that, then modify the answer when testing the others. That can lead to losing information and "popping through" a wall. And the direction in which I pop through depends on which one is tested first.
The conventional gamedev solution to that is to define down the solution set so that it no longer matters which order I test the walls in: with axis aligned boxes, I can say "move only the X axis first, then move only the Y axis". Now there is a fixed order, and a built-in bias to favor one or the other axis. But this is enough for the gameplay of your average platforming game.
The generalization on that is to describe it as a constraint optimization problem: there are some number of potential solutions, and they can be ranked relative to the "unimpeded movement" heuristic, which is usually desirable when clipping around walls. That solution set is then filtered down through the collision tests, and the top ranked one becomes the answer for that timestep.
Problems of this nature come up with resource allocation, scheduling, etc. Some kind of coordinating mechanism is needed, and OS kernels tend to shoulder a lot of the burden for this.
It's different from real-time in that real-time is a specification of what kind of performance constraint you are solving for, vs allowing any kind of performance outcome that returns acceptable concurrent answers.
I bet there are hard-realtime (commercial) OSes running on ARM, and the ability to use a lower-specced (cheaper, simpler, consuming less power) core may be seen as an advantage enough to pay for the OS license.
The power issue is real, but it might well be the same price or cheaper - a standard ARM that gets stamped out by the million can cost less than a "simpler" microcontroller with a smaller production run.
Any system that deadlocks.
Perhaps even that's not an example of such a system, 0.3 microseconds is close to the allowable real-time budget, and QC would probably not scrap a $20k part if you were off by that much once.
But in practice, every time I've heard "soft real time" suggested, the failure mode is not a sub-microsecond miss but a 100 millisecond plus deadlock, where a hardware watchdog would be needed to drop the whole system offline and probably crash the tool (hopefully fusing at the tool instead of destroying spindle bearings, axis ball screws, or motors and gearboxes) and scrap the part.
I’m trying to understand where the roadblock on a rPi + small FPGA hybrid board for $50 fails at the task… and it sounds like the OS/firmware doesn’t suffice. (Or a SoC, like a Zynq.)
Eg, if we could guarantee that the 1.5GHz core won’t “be off” by more than 1us on responding and the FPGA can manage IO directly to buffer out (some of) the jitter, then the cost of many hobby systems with “(still not quite) hard” real time systems would come down to reasonable.
Hobby systems like a small CNC mill or lathe usually don't need anything near 1µs (or better) max jitter. LinuxCNC (derived from NIST's Enhanced Machine Controller, name changed due to legal threats) runs fine on preempt-rt with control loops around 1kHz, with some systems you can also run a "fast" thread with say 20kHz and more to generate stepper motor signals, but that job is best left for the FPGA or an additional µC IMHO.
But it's not really so much about being fast, it's about being able to guarantee that your system can respond to an event within a given amount of time every time. (At least that is how a friend who works in embedded/real time explained it to me.)
If you really need realtime, then you really need it and "close enough" doesn't really exist.
This is just my perception as an outsider though.
Even if a system can run (RT-)Linux, it doesn’t mean it’s suitable for real-time. Hardware for real-time projects needs much lower interrupt latency than a lot of hardware provides. Preemption isn’t the only thing necessary to support real-time requirements.
linux RTOS is at microseconds granularity but it still can not 100% guarantee it, anything in cache nature (L2 cache, TLB miss) are hard for hard real time.
a dual kernel with xenomai could improve it, but it is not widely used somehow, only used in industrial controls I think.
linux RT is great for audio, multimedia etc as well, where real-time is crucial, but not a MUST.
yup that's why you'd pin the memory and the core for the critical task. which, alas, will affect performance of the other cores and all other tasks. and whoosh there goes the BOM...
which again as we both probably are familiar with leads to the SoC designs with a real time core microcontroller and a HPC microprocessor on the same package. which leads to the question how to architect the combined system of real-time microcontroller and compute power but soft real time microprocessor such that the overall system remains sufficiently reliable...
oh joy and fun!
How the CPU and MCU communicate is a good question to tackle, typically chip vendors provide some solutions, I think OpenAMP is for this.
Linux-rt makes linux actually decent at soft realtime. PREEMPT_RT usually results on measured peak latency for realtime tasks (SCHED_RR/SCHED_FIFO) on the order of a few hundred usec.
Standard Linux lets latency go to tens of milliseconds, easily verifiable by running cyclictest from rt-tests for a few hours while using the computer. Needless to say, this is unacceptable for many user cases, including pro audio, videoconference and even gaming.
In contrast, AmigaOS's exec.library had no trouble yielding solid sub-millisecond behaviour in 1985, on a relatively slow 7MHz 68000.
No amount of patching Linux can give you hard realtime, as it is about hard guarantees, backed up by proofs built from formal verification, which Linux is excluded from due to its sheer size.
There's a few RTOSs that are formally verified, but I only know one that provides process isolation via the usual supervisor vs user CPU modes virtualization model: seL4.
https://www.audioadvice.com/audioquest-nrg-dragon-high-curre...
(I didn't know the term, trying to be helpful if others don't)
Depending on implementation, it will either pause or play the old sample where the new one isn't yet but should be.
An xrun typically manifests itself as a (very short) discontinuity or gap in the audio signal. It might sound unpleasant, but there's nothing dangerous about it.
Examples of the latter include audio and video playback and video games. Nobody wants pauses or glitches, but if you get one once in a while, nobody dies. So people deliver these on non-RT operating systems for cost reasons.
Do you really stand by the statement of this rhetorical question? Because if yes: this attitude is a big reason for why web apps are so unpleasant to work with compared to locally running applications. Depending on the application, even 16ms vs 32ms can make a big difference.
No. This is a common misconception. The distinction between a hard realtime system and a soft realtime system is simply whether missing a timing deadline leads to a) failure of the system or b) degradation of the system (but the system continues to operate). Safety is not part of it.
Interacting with the real physical world often imposes “hard realtime” constraints (think signal processing). Whether this has safety implications simply depends on the application.
On the one hand they aren't safety critical.
On the other, I can imagine someone getting chewed out or even fired for a pause or a glitch in a professional performance.
Probably the same with live commercial video compositing.
But there are plenty of performers who apparently rely on Linux boxes and gumption.
1) There is always a possibility that something fails to run by its due date. Planes crash sometimes. Cars won't start some times. Factory machinery makes scrap parts sometimes. In a great many applications, missing a real time deadline results in degraded quality, not end of life, or regional catastrophy. The care that must be taken to lower the probability of failure needs to be in proportion to the consequence of the failure. Airplanes have redundant systems to reduce (but not eliminate) possibility of failure, while cars and trucks generally don't.
2) Even in properly working real time systems, there is a tolerance window on execution time. As machines change modes of operation, the amount of calculation effort to complete a cycle changes. If the machine is in a warm up phase, it may be doing minimal calculations, and the scan cycle is fast. Later it may be doing a quality control function that needs to do calculations on inputs from numerous sensors, and the scan cycle slows down. So long as the scan cycle doesn't exceed the limit for the process, the variation doesn't cause problems.
Much easier to just use an RTOS and save the expensive testing.
Therefore it'd be remarkable not to see a certified RTOS in such industries and applications where that validation is required, like aerospace or medical.
Most software cares about the average case, or, in the case of the Windows 10/11 start menu animation, the average across all supported machines apparently going 20 years into the future.
FWIW by-device/by-transistor-count, the bulk of "hard realtime systems" with millisecond-scale latency requirements are just audio.
The sexy stuff are all real applications too. But mostly we need this just so we don't hear pops and echos in our video calls.
[1] Actually even that's wrong: for sure there are teams of people within MS (and Apple, and anyone else in this space) measuring latency behavior at the top-level app layer and doing tuning all the way through the stack. App latency excursions can impact streams too, though ideally you have some layer of insulation there.
Even hard-realtime systems have a failure rate, in practice if not in theory - even a formally verified system might encounter a hardware bug. So it's always a case of tradeoffs between failure rate and other factors (like cost). If commodity operating systems can push their failure rate down a few orders of magnitude, that moves the needle, at least for some applications.
But it all depends on what your application is. There are a lot of applications that are "almost real-time" in need. For those, this might be useful.