Comment by meesterdude | Hacker News Reader

Comment by meesterdude | Hacker News Reader

Parent Full thread

So, take this into account, and add in self driving cars. The complexity involved in these, and the coordination required, is nontrivial and its clear from the article they do NOT have a good handle on things.

I mean, most cars do seem to get around OK, so I'm partly amazed that under the hood, things could really be so scary. But we can't be lazy about adding complexity and systems, especially as more faith is placed on them.

That this is not NASA grade code (or anything close) does not give me any warm fuzzies. It reeks of unprofessionalism, greed, and laziness.

There needs to be a sane-software assesment. People rely on products with an ever increasing amount of source code - it would be interesting if a third party could come along and certify that a given codebase is not a big scary unmaintainable mess; not to say there won't be bugs or issues, but that these guys are at least trying to make a well designed system, and aren't doing a bunch of crazy stupid things.

The companies currently most actively involved in automated automobiles (hah) are pretty good at what they do. Volvo has been producing big trucks for several years now that have remarkable safety features, like automatically detecting stopped traffic ahead and bringing the fully loaded truck to a complete stop in what seems like an impossibly short distance (https://www.youtube.com/watch?v=HoCknasKdRU).

There are plenty of things I dislike about Google but I can't say they're bad at software. If anybody can build a safe automated vehicle, they can.

Tesla, too, who have a lot riding on their hard-earned reputation for building next-generation vehicles.

I'm an old-school car guy, I haven't been a big fan of the complexity added to cars ever since the really bad emissions control systems in 80s cars, but even I have to admit that all that complexity has probably saved a lot more lives than it has cost -- antilock brakes, airbags, better pollution control, better fuel economy, way better structural safety features.

If you've ever driven a vehicle with them you'd know that air brakes are very, very powerful.

There are standards, Toyota simply chose to ignore all of them. Standards such as MISRA [1] and DO-178C [2] exist for the purpose of ensuring software quality in safety-critical situations. Most embedded software development environments even include tooling to help verify that you're not doing the things that are unsafe. The problem lies more in that automakers aren't required to use any such standard, unlike what the FAA has required for decades.

[1] http://en.wikipedia.org/wiki/Motor_Industry_Software_Reliabi... [2] http://en.wikipedia.org/wiki/DO-178C

Toyota couldn't ignore these (voluntary) standards given that MISRA-C nor the earlier DO-178 didn't exist at the time the code was developed.

MISRA-C dates from 1998, and DO-178B, which was the previous revision of DO-178, dates from 1992.

Yeah, I mistyped (should read "...later DO-178..."). You're quite correct that DO-178B was around then, but the C revision was not.

According to testimony, Toyota's coding standard was in place in 1997, before the first MISRA-C publication.

First, you're ignoring the words "such as".

Second, DO-178 was originally published in 1992.

Very awesome comment! I didn't know there were standards, but it's great to see there are. However, Toyota should have followed them. I mean, did they think they knew better? I think with self driving cars, the standards will become more requirements; though I doubt before a series of deaths result from software failure.

Given that a lot of computer vision work is based on randomised algorithms, do you think that these standards would be enough? You could demonstrate 100% MC/DC coverage through a neural net implementation, but the weights are where the faults probably exist, for example.

That actually leads to a very interesting point; would self driving cars be vulnerable to adversarial imagery?

Neural nets are well known for being easily fooled [1] ... I wonder if you could create similar situations for self-driving cars.

[1] http://www.evolvingai.org/fooling

Deleted comment

Shouldn't toyota be closer to NASA than google? I mean, we're talking about systems that peoples lives depend on. I would be OK if my website was built by some IT department, if it meant that some level of NASA engineering is going into the control systems i rely on in my day to day life. That this is not the case is a little surprising to me, to be honest.

Self-driving cars seem less scary to me. You can take unreliable components (like, say, a car with crappy firmware) and make a reliable system out of it if the software running on top is fault-tolerant. The "autopilot", unlike a human, will pull over the moment the car's brakes begin to fail, etc.

Reply on news.ycombinator.com