Comment by dsuth | Hacker News Reader

Comment by dsuth | Hacker News Reader

Parent Full thread

> But the idea that code applying to the primary control functions of an automobile cannot be known (is not published and cannot legally be reverse engineered) is just a bad idea.

Why is it a bad idea? Do we know the code base of airplanes? Critical infrastructure, like power and water plants? How about military software that controls missile guidance?

The answer isn't to open source everything and let programmers sort it out. We have regulatory and safety boards specifically to counter the issues around public safety that software in critical applications causes. A huge amount of time and money is spent developing standards, and verifying and monitoring compliance with them.

Obviously these processes are not always perfect. In this case, it will be interesting to see how far the corruption necessary to include a pollution-defeat spreads. But throwing out the whole process and just publishing code in its place is not a reasonable solution. More stringent black box testing by experts could have caught this issue far sooner.

>Why is it a bad idea? Do we know the code base of airplanes? Critical infrastructure, like power and water plants? How about military software that controls missile guidance?

I OWN my car. I do not own nukes and powerplants yet. My evil genius lair is still under construction.

Ownership should come with reversal rights, rooting, reflashing, modifying etc for this one unit I bought.

By purchasing a car, you purchase an end product, which is designed to be suitable for your purposes. You don't purchase rights to a million manhours of software, free to do what you will with. If companies had to amortise the cost of code development over the cars they sold, you would end up paying a lot more for 'your' car.

Actually I do purchase it. I don't purchase the right to redistribute it. But I totally own the right to modify my car however I see fit as long as it is road legal. Part of it being mine.

Are you serious? Absolutely not. Unless you can show that you are competent to modify safety critical software, and have a certified process in place to do that, it would be illegal for you to modify the code, and you would be personally liable for any accidents caused by such a modification.

You would also require the full lifecycle documentation to allow you to understand the impact of any modifications you make, and be required to do a full impact analysis to prove that any modifications you make do not reduce the integrity of the existing safety functions.

That's completely ignoring the vendor's configuration management requirements (which you can't do).

This the whole point - devices run by software systems are too complex to be modified by a layman. There are very detailed, statutory processes and requirements around the development and modification of software in safety critical applications, and you absolutely cannot modify it just because you bought it.

>Unless you can show that you are competent to modify safety critical software, and have a certified process in place

> require the full lifecycle documentation to allow you to understand the impact of any modifications you make, and be required to do a full impact analysis to prove that any modifications you make do not reduce the integrity of the existing safety functions

Car manufacturers does not use formal verification, even though it exists, and would be able to give hard guarantees about safety and the like. And given recent history about analysis of code that resulted in run away bugs, I, as a professional developer, are completely confident that few if any manufacturers do the above. They have an extensive testing procedure, surely, but they're not trying to avoid the bugs earlier in development, nor try to enforce a coding style that reduce the risk of bugs.

But besides that point, many people are not arguing that they should be allowed to tinker with safety settings and drive on the road. That would be illegal, just as it is illegal to remove the lights and drive at night. But I as an owner of the car, should be able to see and change that code for auditing purposes, or use on a closed road. If the entire system of the car is open, it is also trivially easy to compare the running code with the version supplied from the manufacturer and see if any modifications have been made.

European car manufacturers are required to develop safety critical software under ISO 26262, which is a derivation of IEC 61508, which absolutely does require formal verification and validation activities.

If you change the code outside of the development process, you could unwittingly compromise the safety of the vehicle. The manufacturer is required to use access controls to prohibit people from changing the software for exactly this reason.

ISO 26262 does not to my knowledge require formal verification, and some googling around seems to support this. Without access to the actual specification I cannot find out exactly what it requires.

So you don't know what's in the standard, but you make assertions and continue to support them? That's a fairly disappointing level of discourse for HN. It requires a very similar software development process to all other functional safety standards, in which verification and validation are key steps.

Here is a paper from Mathworks describing verification and validation according to ISO 26262:

http://www.mathworks.com/tagteam/71300_1D-4.pdf

I know how ISO standards are implemented in two unrelated fields, that was really the basis for my comment. Besides, I am now certain that it does not require formal verification, as several companies sell products that support formal verification as a mean to pass the verification part of the ISO.

I'm not going to pay for access to the standard just for a comment on HN.

When you say European manufactures are required to follow this, what about non European manufactures?

Actually that's an interesting point you raise. The automaker provides you with a finished car containing finished machine code. It doesn't provide you with the CAD files needed to reproduce the car, the techniques used to built it, or the source needed to reproduce the machine code. And maybe they shouldn't be required to.

That said, I think it's reasonable to reverse engineer the code, just like you could physically take apart the car to figure out how it works. As long as you don't disseminate the results of that work, it shouldn't be illegal in my opinion. Even dissemination should be legal in some circumstances, like whistleblowing.

It's a bad idea because it's a departure from our historic common sense, as well as patents and copyrights. It's commonly accepted that when you buy a car you can learn how it works by poking it with a stick, and make modifications, and everyone expects that the things they modify they assume liability for and a warranty exemption. Code? Nope. You do not own that part of the car. You don't have the right to figure out how it work, or modify it, or even replace it. There's residual ownership in the form of IP, in perpetuity.

One of the major purposes of patent law was to make the entire details of an invention public so that it was possible to know exactly what monopoly was being granted, and also permit people to learn and invent something better. That doesn't happen when things are obscured like this.

And just because airplane code isn't published doesn't mean it's a good idea.

Reply on news.ycombinator.com