Toyota's killer firmware: Bad design and its consequences (2013)
edn.com
edn.com
>> Toyota loosely followed the widely adopted MISRA-C coding rules but Barr’s group found 80,000 rule violations. Toyota's own internal standards make use of only 11 MISRA-C rules, and five of those were violated in the actual code. MISRA-C:1998, in effect when the code was originally written, has 93 required and 34 advisory rules. Toyota nailed six of them.
How the ACTUAL FUCK did this happen!? The article makes Toyota's engineering team seem egregiously irresponsible. Is it typical for vehicle control systems to be this complicated? I would love to hear the other side of the story (from Toyota's engineers). Maybe the MISRA-C industry standard practices are ridiculous, out of touch and impractical.
The limb is: don't let EE's lead a firmware group.
It's a natural division of labor. The hardware guys are familiar with all the component datasheets and the bus timings and all the other low level details. If the prototype is misbehaving, they'll grab a scope and figure it out. They're probably the only guys who can. Software is an afterthought, a "free" component.
Those same EE's who have performed that role get promoted to buck stoppers for hardware production. They're familiar with hardware design, production, prototyping, Design for Manufacturability, and component vendor searches. They might cross over with production engineering and that six sigma goodness. They're wrapped up in cost-per-unit to produce which is direct ROI. Software remains a "free" component; you just type it up and there it is.
The culture of software design for testability, encapsulation, code quality, code review, reuse, patterns, CMM levels, etc etc is largely orthogonal to hardware culture.
I find we can reliably write small programs that will do everything they need to. But as the program grows, we are not adept at managing the explosion in complexity.
However I don't believe that's because hardware culture doesn't value software or anything like that. We just have no proper education in software engineering.
Speaking for myself, I grasp most fundamental software concepts. Memory structure, search algorithms, that sort of thing. I appreciate the value of abstract code qualities like testability, simplicity, reusability, etc.
I just have no actual education in how to design large programs from scratch that will achieve those goals.
I know some basics of EE but I have no gut feeling of how to build or analyze any hardware for real. It's just something I've read and made my mind to understand but it's not something I know.
Conversely, I've been writing software on the lowest to highest levels for decades and I have a hunch of how to start building something that will eventually grow really big and complex. I don't know exactly how I do it but depending on what I want to achieve I have, already at the very beginning, a quite strong sense of what might work and also what will definitely NOT work.
My take on hardware is that it's mostly a black box that usually does most of what's advertised (but workarounds are regularly needed, and I guess it must be really difficult to build features into silicon and have it work 100% as designed), and these quirks had better be encapsulated in the lowest levels of the driver so that we'll get some real software building blocks sooner.
This would be no basis to design hardware. I would make such a terrible mess out of it that even if I managed to design a chip and make it appear to work somewhat, it would fail spectacularly in all kinds of naive cornercases that a real EE would never have to solve because s/he would never venture to build any of them, just because s/he would know better from the start already.
Corollary: Don't let EEs hire your firmware engineers. They have no idea what to look for.
I'm confused by this statement. Substitute say ASME for CMMI and the rest of this sounds like good engineering practices in general. What, for example, about "hardware culture" is orthogonal to testability?
Software only needs testing when the tools or inputs change. So design for testability of software is driven by the needs of a development group (eg simulators, debug statements), while hardware (at a minimum) is aimed at a post production group (eg JTAG, test fixtures).
I agree there are many parallels, but I think the culture is different.
I currently work in the static analysis industry. Broadly speaking, the MISRA-C ruleset is, in my experience and in the experience of many customers who apply it, not impractical, clearly of benefit, and the majority of rule violations (including a number of rules violated by Toyota) easily found with static analysis tools.
I even know that Toyota is a customer of (at least) one of the static analysis tool companies (obviously Toyota is a huge company, and I've no idea if the specific muppets making this clusterf had any static analysis tools; just that Toyota as a company definitely has someone buying them). It seems that they either just didn't use it, or just didn't care (or were told not to care).
As an extra point of data, I sit opposite someone who is on the MISRA-C committee. He is a solid C coder who knows a great deal about the language and how to get it wrong. His day job is writing/maintaining static analysis tools for C programmes. Obviously this is no guarantee that the committee as a whole is good at maintaining the MISRA-C ruleset, but they do have at least one active, experienced and competent C coder (with lots of experience of having to actually automate detection of rule violations) at the table.
On top of that car companies are massive and capital intensive with all the lack of agility that requires...retooling a production line takes many years from design to cars at the dealer. The critical decisions about programming stack had to be made in the early 1980's when the first microprocessor based control systems appeared in automobiles.
Back then, automotive engineers and executives would have hardly predicted the software complexity that was on the horizon. It was state level actors who came up with Ada and Drakon...But those are money is no object and failure is not an option inspired. Toyota probably made money via their level of software quality...shutting down production for a year to get it right would have cost billions. It followed the bean counting and rode the tiger.
I remember when nearly everything came with schematics so it could be fixed.
Me too, it's probably indeed bad, but there's ALWAYS another side to such stories. Unfortunately, we rarely get to hear those.
Early firmware was a replacement for analog electronics. Consider modeling the classic lunar lander. A simple digital computational loop with no branches, just math, has advantages over analog circuitry, such temperature stability, noise immunity, reproducibility, etc.
From there, embedded firmware grew in the same way complex mechanical systems grew, like mills and robots. Meanwhile, non-embedded software on workstations was growing, theories were developing, algorithms and organization and business models were applied. And firmware grew and grew, but mostly like an engine grows in that abstractions are minimized, optimization happens in-place and theory is largely tribal knowledge.
It's only in the last 10 years or so that firmware has metamorphosis end and is confronting other software. It looks especially conflictual (word?) to high-level devs like Ruby or what-have-you.
So yeah, this is no excuse for negligence and ignorance. I just hope my perspective helps ppl evolve.
http://en.wikipedia.org/wiki/Apollo_Guidance_Computer#Softwa...
In addition, there is no way in hell that the control algorithms it was using could have been developed without the use of computers. State-space control theory was specifically developed to take advantage of discrete-time control systems.
I'm a bit puzzled by this figure given that according to the article the typical LOC count of this kind of software is within that order of magnitude ("tens of thousands of lines of code").
For example, the static analyser I'm working on finds 32700 violations of MisraC2012 in the SQLite codebase (130 kLOC).
This is because Misra is quite strict about the use of C: a simple statement like `if (ptr) f();` already violates two rules:
* use explicit `ptr != NULL` comparison
* always use braces with `if`
Once you get around to integer arithmetic, where MISRA is quite strict about not relying on implicit conversions or numeric promotion (which is non-portable due to depending on the size of `int`), it's not rare to have 5 or more violations in a single line of code.
I'm also surprised C is still so commonly used for mission critical software. I understand that C is familiar, has many static analysis tools (to make up for the language's deficiencies), and has a straight-forward translation from C to object code (though only when using simple optimizations). For example, if MISRA-C's coding guidelines disallow recursion, why not design a language that only supports DAG function dependencies?
Also on the horizon is ISO26262, which mandates quality assurance for automotive embedded code in the form of paper trails. Unfortunately due to the huge amount of work required by the standard, some automakers are choosing to ignore it and hope it doesn't become mandatory.
Personally, I could never handle the stress of designing or building something like this... I worked in a security software team for a bank for a year, that was about as far as I can get in terms of job stress. Just the same, it seems to me some of these choices were clearly rookie level mistakes that shouldn't have been put in production to control motor vehicles.
I also agree, that it would really make more sense for car mfgs to get together to form some standards for production and core controls. If they're determined to use an actual OS, then it better damned well be the most thoroughly tested OS, running on certified hardware, with certified controls in place.
If you mess up the code in a number of places, people don't die... Tasks that control fast moving and/or heavy machinery, medical devices, etc should have very tight controls in place.... this is the kind of situation where software is and should be treated like an engineering discipline. Most of the time it's more of a craft, this isn't one of them.
If you're surprised at that then you probably have not seen much of the embedded world where (gasp!) assembler is still in use and C is considered a 'high level' language.
This latter feature is almost unthinkable in what most people would call high level languages today (Java, C#, Swift, etc).
For better or worse, Java is increasingly being used in embedded scenarios. Blu-ray players are a prime example, but televisions, settop boxes of various kinds also use Java these days.
[1]: http://en.wikipedia.org/wiki/Esterel
[2]: http://www.esterel-technologies.com/products/scade-suite/
Ultimately, it's impossible to know without being inside. I would imagine there's a fair bit of reuse within each manufacturer and very little sharing between manufacturers
1) how much other components are shared between manufacturers, and having companies like Delphi Automotive doing a lot of work for many manufacturers
2) how often car manufacturers own parts of each other, and how often these ownership structures change (look at Ford and Volvo and Mazda, or Daimler-Chrysler, etc)
I would imagine there are shared platforms and components, also software, between manufacturers.
One item I'd like to point out is that Mr. Barr criticizes the lack of hardware logic to close the throttle if the driver rides the brake.
In 1987, BMW introduced the 1988 model year 750i, with a V12 engine that has two intake manifolds and two throttle valves. The engine controls and electronic throttle system was made by Bosch. Whether the logic is in hardware or software, I don't know, but it doesn't take long for it to slam the throttles shut if you hold the brake pedal down. When this happens when the engine is delivering power, it's a very severe shock as engine power is removed and stops working against the brakes.
Due to the time it takes to go to manufacturing from the design stage, obviously the Germans had this figured out in the middle of the 1980s decade. BMW went on to use the electronic throttle system in a bunch of 90's model 5 and 7 series cars with normal six cylinder engines. The only problems that I know this system, called EML, caused were for owners and technicians who did not understand how the system works. In other words, when this failed, the result was that the car would not go anywhere.
And a colleague of mine with a late 90's Volkswagen also proved that the throttle slams shut if he rides the brakes while requesting engine power with the gas pedal.
Mr. Barr points out that in 2005, the Camry had no such logic. I seem to remember a youtube video where Consumer Reports guys test this out in some kind of Toyota, and they could have gone all day long until the brakes melted.
Toyota did not include this logic for safety even though it had been in cars released to the market decades before. The ways that Toyota handled this situation, everything they've done from blaming the operators, the potientometer supplier, the floormats, to their brazen delays in producing discovery, reinforce my bad experiences owning a Toyota and my conclusion that they are a bad actor.
Maybe automatic cars do this but when driving a stick shift a common advanced driving technique is to heel-and-toe on downshift so that you can rev up the engine to the correct RPM to not upset the car. That requires revving up the engine under braking. As far as I know this is still possible in most cars.
The usual guarantee is that the brakes are always specified to be able to overpower the engine even at full throttle. So even if you have a stuck throttle for some reason you should always be able to safely stop just by standing on the brakes, even if it takes you a little longer. Naturally in a manual car you should just stand on the brakes and clutch for emergency braking and then the engine is completely disconnected no matter what the throttle is doing.
the technique being mentioned is more akin to 'left-foot braking', a technique used to pivot the weight balance from the rear to the front in order to induce certain driving characteristics that may be beneficial in a turn. It is a common technique to balance out the inherent understeer of a front-wheel drive car to a more neutral balance mid-turn in an effort to reduce lap times. It's quite common in rally.[0]
and as was said, many cars disallow left-foot braking now; with the worst responses triggering semi-permanent CELs (CELs which require mechanic intervention, as opposed to being clear by more drive-cycles).
Probably not a bad idea, as it's an advanced technique that quite easily upsets a car mid-turn, and is often just compensatory for an ill-setup race car.
The technique is used in racing during corner entry to allow the driver to rev-match downshift while under hard braking. This way they are already in the optimal gear as they release the brakes and initiate the turn
Both heel-and-toe and left foot braking require revving up the engine under braking. The difference between the two is if the car is in gear and the clutch engaged at the time. Maybe modern cars have enough sensors to know if the engine is actually driving the wheels so they can disable the throttle under braking in those situations.
This is possible, any car with traction control has wheelspeed sensors for all wheels, and manual shift cars have a pushbutton switch on the clutch pedal arm (for enabling the starter, or for a car with electronic throttle, for cancelling the cruise control.)
I don't know how racetrack drivers do this, but when I downshift, the crankshaft has some opportunity to fall a little while I get the gearbox in neutral and engage the clutch again. Then the crankshaft speed rises to 4000 rpm or more.
My friend's VW was a manual shift, some variant of Golf, and he did encounter this behavior on a track day.
So it looks like VW was rather thoughtful to introduce this delay, when there is a manual gearbox.
Wouldn't the left foot braking be more useful in a car with a really light front end, like MR2 or other mid-engines?
[Edit: My 5 series is a manual shift with traction control, with a v8 engine that never got electronic throttle on this series. There is a second throttle plate ahead of the usual throttle that the traction control will close during intervention.]
[1] https://www.bmwtechinfo.com/repair/main/741en/images/7410265...
Regarding your second point, similar complaints were made about other safety features, like mandatory turn signals or airbags. Yes, there will be additional time spent ensuring that they are compliant with safety standards. But in this case, a lot of that time should have already been taking place. Some code needs more review time, a second or third or even fourth set of eyes on it. When people can die because of a bug, then the bar for what is acceptable code can and should be higher.
your statement sounds kind of contradictory to me. If there many - just choose any path. It may be impossible to say which one actually happened, yet should be possible to show an actual path, at least one, that could plausibly lead to UA.
I think the answer is no -- there was never a clear demonstration of an extant buggy execution trace. Just a preponderance of evidence that such a trace could exist.
http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf
For example, we observe DRAM error rates that are orders of magnitude higher
than previously reported, with 25,000 to 70,000 errors per billion
device hours per Mbit and more than 8% of DIMMs affected
by errors per year. We provide strong evidence that memory
errors are dominated by hard errors, rather than soft errors, which
previous work suspects to be the dominant error modeI recent called our city's public works dept to propose changing the traffic signage/signals at an extremely confusing and potentially dangerous intersection (every person I've talked to about it hates making the turn there...). The lead traffic engineer was very understanding and agreed it was a terrible intersection. But the hard stats showed few accidents there, so he'd never be able to redirect money from more problematic areas...
I can't currently find it, but I once read about a roundabout that had accident problems, but had very good signage. People felt entirely in control going into it and weren't paying as much attention as they could be. By removing the signage people wouldn't inadvertently take for granted the behavior of the other drivers, they'd slow down more, they'd pay closer attention, and as a result the accident rate dropped.
So, to answer your question, maybe they do, but probably not by necessity.
Big corporations thrive with complex standards being imposed by governments, because it increases the barrier to competition. $50m in compliance costs might hurt Toyota a bit, but make the next Tesla unfeasible to get off the ground.
Therefore, they'd most likely see it as a positive, even if some government employees got to see their trade secrets and the potential for leaks.
On the other hand, there's no upside to aftermarket hacking of your product, and plenty of downside (from the corporate perspective).
Need to focus where it actually makes a difference.
I fear for when I have to upgrade someday to all drive by wire.
As coders we all know how bad some code can be or even how the best code has flaws.
It's not recursion that is bad, it's unchecked stack growth, in environments where beancounters made sure the hardware is adequate but not more...
http://ridiculousfish.com/blog/posts/will-it-optimize.html
C the language does not specify how optimizing compilers should handle this situation. -O2 on gcc will get you this optimization (and many more), it will even do TCO on really hard cases, some where FP languages/compilers would fail!
That said, writing code that fits the construction of the language is probably more useful than siding between "loops or recursion". If I'm writing Scala, my code is usually chock-full-of-recursion. If I'm writing C, not really as much, because I'm thinking "which bits go into which memory". With Python I tend to think in terms of list comprehensions, etc.
Recursion is A BIG ERROR for mission critical software because it needs variable memory in the stack that depends on the implementation and depends on the other memory on the stack(depending on where it is called), creating stack overflows, breaking completely the system.
This means that you write software today for a car, it works fine. Tomorrow you reuse it for a motorcycle and only on specific cases it breaks dramatically, because the motorcycle people used another debugger or another OS with different assumptions.
Now when you debug something, you expect errors, and errors in the new code. But when something already runs and fails sporadically, it is extremely difficult to isolate the bug, because you can't see the bug but the cascade consequences of the bug.
"What if the solution using recursion results in more readable or declarative code?"
Recursion is something that looks perfectly ok in code,on paper could look gorgeous, but could fail terribly in the implementation.
The judge is not interested, when you say to her: yeah, it broke and killed someone, but it looks ok in the code!
"(especially with languages/compilers that do tail-call optimization properly?)"
If compilers were perfect, it will be no problem. On real life no compiler is perfect.
We design compilers and they are pretty stupid, it is really hard to represent all the complexity of the world for any given possibility in a compiler.