Volkswagen and Cheating Software
schneier.com
schneier.com
Can you imagine this happening with, say, an architect? When unsafe buildings fall down, it makes international news. By and large, professionals have a code of conduct which they must follow or there is a very real chance that they will lose their livelihood.
I'm a Member of the British Computing Society - it has a fairly simple code of conduct for members. http://www.bcs.org/category/6030
I can certainly see adding this functionality would probably be a breach of ...
> 1 a) have due regard for public health, privacy, security and wellbeing of others and the environment.
> 2 d) ensure that you have the knowledge and understanding of Legislation* and that you comply with such Legislation, in carrying out your professional responsibilities.
> 2 f) avoid injuring others, their property, reputation, or employment by false or malicious or negligent action or inaction.
> 3 e) NOT misrepresent or withhold information on the performance of products, systems or services (unless lawfully bound by a duty of confidentiality not to disclose such information), or take advantage of the lack of relevant knowledge or inexperience of others.
But, here's the kicker - if I were kicked out of the BCS for adding this code, nothing would happen to me. Employers don't care about professional bodies - except in terms of certification and, possibly, indemnity.
I'm quite happy being a member of a Trade Union, because I believe it offers me the best protection against a malicious employer - I wonder how long before more codes start joining professional bodies to help protect themselves from being asked to act counter to their best interests?
I agree with the spirit of the rule, but I don't think it makes sense as it's basically asking all programmers to become lawyers. It should be more along the lines of "comply with legislation you are (made) aware of" and require some due diligence like "solicit the services of a lawyer if there is a reasonable belief there is legislation that could apply".
I honestly don't think you need a legal degree to know that "dramatically alter how the pollution controls works when in a test environment" is likely to be extremely dodgy!
Now, maybe these programmers did flag this as a problem. And, maybe VW's high-priced lawyers said it was OK. In which case, responsibility is probably shifted.
I'm not saying every programmer should be a lawyer. But you can't say "how was I supposed to know that running a red light while drunk is illegal?! I'm just a delivery driver - not a lawyer."
They also have a number of reports & white papers which members can read.
Members also have access to a legal helpline. Don't think I can link to it, but it says...
> As part of the ongoing commitment from BCS to invest in services and benefits, the BCS has launched a new and improved Legal Advisory Service comprising two new online legal information websites and a dedicated telephone helpline for both Business and Personal legal issues (restrictions apply*). These services are totally FREE and are provided by Law Express, the UK's leading specialist provider of legal advice.
I've not had cause to use it, so I can't comment on how well it works in practice.
Holding the P.Eng. designation myself I would be liable to lose my license had I published this software, I really hope we can reach a point in our profession where a higher standard of care to public and environnment is not only expected, but enforced.
For software it's hard to see where to draw the line. For other engineering disciplines this can be pretty easy. I can release a little mechanical toy and not be an engineer. The public won't try to use it to drive around town.
But if I make a pet software project millions of people might decide to use it to "drive around town", and some portion will hurt themselves.
I think that if you build stuff for certain things (such as 1 ton objects zooming around at 70+ miles per hour), it's obvious. But what about financial software? What about social networks (privacy issues)?
When I first started programming I built a D&D utility for a friend. He passed it to his friends. Soon around 10 of them used it. If it had a serious bug, should I have been prosecuted? Should I have been unable to build it for him in the first place?
If you say all of this activity should be restricted, how do you get from here to there? Some of the best "software engineers" in the world aren't technically software engineers.
Software developed to control an automotive emissions system by a manufacturer and in the context of being used for that purpose would not be able to escape that.
> But if I make a pet software project millions of people might decide to use it to "drive around town", and some portion will hurt themselves.
That is a legitimately unique aspect of software and deserves consideration.
> I think that if you build stuff for certain things (such as 1 ton objects zooming around at 70+ miles per hour), it's obvious. But what about financial software? What about social networks (privacy issues)?
Likely some portions of the system matter and some don't. Which ones likely require engineering judgement. For physical products it's pretty easy to make that call for a paper weight and for an airplane, but all the products in between need that decision. We are generally building software systems that are about as complex as a Black & Decker power drill. When designing something like that you need to make damn well sure you are not electrocuting the user, but if the gears break and it doesn't function you aren't hurting anyone. Therefore most drills are 'double insulated', but there is no backup for the actual function. In software we should make that same call. What is the consequence of failure? If it's small then we live with it. If it's not, then we should build a fail safe / backup system.
I'm super in favor of software engineering licensing and all that it entails, but I do think large swathes of software are completely immune because failure has no consequences. Your D&D utility is an example. Most open source licenses rightly have a clause that should protect them from civil lawsuits, much less prosecution.
This is where software seems unique - this seems false to me. A common failure mode of software is code execution, which has nearly unlimited consequences.
E.g. my D&D utility could read and write files. But it might have a bug in the file reading code (Edit: or anywhere in the code), which someone could exploit with a properly crafted file to gain access to the user's computer. But you said this utility was immune to failure. It seems clear to me that it isn't.
There is no physical object analogy here that makes sense to me. It would be like - if you used your black and decker drill on a piece of wood someone gave you, it could electrocute you if there were a flaw in the drill bit. Or it could turn off your refrigerator. Or it could play a song to you. This makes no sense, but this is the world software lives in.
Edit: or do you mean no physical consequences, so only software controlling physical devices require the proper licensing?
But you do raise a point about the real costs of mistakes even in trivial systems. Arguably that makes the point that operating systems should have stronger sandboxes between apps. An operating system is effectively infrastructure, you probably want higher quality standards there.
> Listen up, this will be on the test. You need to help patients, not hurt them. You need to be responsible. Try to avoid being irresponsible. Try to act professionally, and to be a positive asset to society. Don't abuse nurses. If someone asks you do to something unethical, tell them you won't. Did I mention you need to help patients, and not hurt them?
> If reading that paragraph seemed like a waste of your time, imagine how I feel after attending the three hour lecture version.
Engineering ethics would be a similar waste of time. Good people don't need courses to know that cheating emissions tests is wrong. Bad people won't be swayed by a three hour lecture.
Here's an idea that would be more effective at reducing future harmful software: Run git/svn/hg blame to see who wrote the emissions-dodging code. Then arrest those people and charge them with crimes.
1. Against medical ethics http://squid314.livejournal.com/285785.html
But the point still stands, that there should be some coordination among programmers to ensure that e.g. when someone is asked to do this, they have a group to fall back on, who's willing to help him/her out and boycott this employer.
Software engineers who are any good at all can find work easily. They are among the last people who should feel held hostage by a job. If your personal ethics are not enough, I don't see how a group will help.
Try that attitude on for size if you have a family that depends on your income, or you live in a city that isn't a tech hotspot, or you or your spouse or children have medical issues and depend on your medical insurance. You can't pay the rent or mortgage with ethics. There's lots of towns even in America where there are only a handful of high tech employers so no, you don't have alternatives. Unless you consider moving, which can easily cost $15k in moving costs and tens to hundreds of thousands of dollars in housing costs to be free. Not to mention the disruption to friends, family, and social networks caused by moving. Or care arrangements -- grandparents babysitting is close to free; daycare can easily be $400/week/child (or $600/week/child for under 2 y/o) in sf.
- what should you do if your government demands you secretly backdoor a specific user?
- what if your easiest recruiting channels yield a higher proportion of white males than other, costlier recruiting channels do?
- if your startup is struggling to even survive, should you invest time in infrastructure that protects users' privacy from your employees?
- how do you find nonprofits and companies that are aligned with your values and need your help?
Those are just four questions off the top of my head. I think there are plenty of thugs to talk about to fill a semester, let alone an hour.
They've got a warrant or a court order? OK.
> - what if your easiest recruiting channels yield a higher proportion of white males than other, costlier recruiting channels do?
If you're a good person, you try to correct for that. If you're a money-grubbing sociopath, you try to correct for that. This may sound unusual, but think about it from the sociopath's point of view: If other companies are neglecting some potential candidates, you can hire those people for less pay. Recruiting is cheap compared to salaries, so it's worth the upfront cost. In the long run, even if everyone is a money-grubbing sociopath, all demographics end up being paid the same.
> - if your startup is struggling to even survive, should you invest time in infrastructure that protects users' privacy from your employees?
Probably not. A dead company can't defend users' privacy very well. Also, assets (such as user data) tend to get sold off when a company shuts down. Best is the enemy of good, and all that.
> - how do you find nonprofits and companies that are aligned with your values and need your help?
Givewell.[1]
These are not hard questions to answer. More importantly: Your hypothetical course, even if taken by every software engineer, would not have prevented the Volkswagen emissions cheating. That was obviously wrong, but the people involved didn't care.
If you have to take a class on ethics, you don't have them.
(Q: What's the "Louis Armstrong Rule"? A: Man, if you have to ask, you'll never know.)
I fail to see what your point is.
If you don't have enough ethics to figure it out yourself, the class is just teaching you how to pretend to have ethics. And like jazz, you will fail just as soon as you are called upon to improvise.
But your ability to be ethical in a specific situation depends on your database of "ethical considerations" about possible decisions you might make that will have serious ethical ramifications and how those things are wired up. The purpose of a course is to build up that database.
You might have a deep and earnest desire to be kind to someone, but without a cultural framework for understanding their feelings, for example, you could be completely unable to do that.
The point is institutional protection for engineers who are ordered to do unethical things and refuse.
I think the VW case can be partially solved by handing out indictments to everyone involved, CEO down to engineer. Working in a corporation does not excuse criminal behavior.
Similarly, I don't think making Financial Engineer a thing would do anything to head off the next financial collapse and reign in excessive risk taking on the part of the banks.
I'd rather see us abandon the bizarre notion that education is in any way a parental or local responsibility. It should be federally controlled with uniform quality for every student and differentiations that make sense (on natural talent, effort, and specializations chosen) rather than those based on parentage.
Good education gives you a better understanding of what you're doing and the context of the world, people, and systems you're doing it to. Of course it makes a difference, and it should be available to everyone.
The difference between someone with a college degree and without is the work they put in. That seems like a pretty reasonable thing to base a hiring decision on. Your suggestion is like saying 'you shouldn't be able to discriminate based on work experience'. The difference is college is 4 years of reading books and taking tests vs 4 years of writing code.
You can argue which one is more valuable / relevant / useful, but they're just different kinds of experience. Not allowing that in a hiring decision is flat out unworkable.
College isn't just a matter of putting in work. It's a matter of putting in money. That is the clear dividing line versus work experience.
http://www.ordemengenheiros.pt/en/about-us/colleges-and-fiel...
However, since the law doesn't require informatics to be inscribed for daily work, most don't.
There may well be valid reasons for the software to vary engine parameters test mode; maybe to compensate for the above.
Then sometime later someone said "This NOx Reduction module is killing fuel efficiency on the open road, how about we turn it off"
The net result is the same but motives and intentions of the individuals are important.
Do we know that the code was written by hand and not auto-generated based on real-world testing/calibration?
Do we know that a programmer has final say on the code that is shipped?
Are you sure management couldn't select from a number of programs based on the criteria they like, without needing additional input from a programmer?
Why didn't management review the codes and put in place procedures to ensure the codes reviewed are the ones actually being used?
Why didn't management implement testing procedures so that the reviewed codes could be verified?
Do you have the specs/requirements the programmers were given?
Edit: Really? Downmods?
Unless there's direct evidence implicating Bosch, then blaming them for the cheating is like blaming a bullet manufacturer when you get caught shooting people.
Bosch might have sold them the pistol, but VW is still the one that shot itself in the foot with it. But was the pistol pre-loaded with toe-seeking bullets, or did VW carefully aim at its own extremity before firing?
There is still some question in my mind as to how complete a defeat device package Bosch shipped. If there were several separate functions like LowestEmissions() and LowestFuelConsumption() and LowestMaintenance() and BestPerformance() and LowAmbientTemp() and HighAmbientTemp(), that's quite a bit different from DetectEmissionsTest() or OperateAsDefeatDevice(). Who was it that wrote the code to switch modes when an emissions test was detected?
So i think there's been a lot of assumption that the software engineers made this code explicitly for production. For all we know they were making what they thought was code for specific in-house experiments and a subset or different set knowingly kept it for production.
In other scenarios, yeah it's perfectly possible for an developer to complete a task that seems innocent enough and unbeknownst to him will later become a part of something nefarious.
Nothing went wrong, the code did exactly what it was designed to do.
Due to manager and business pressure to make more money, multiple sections are subverted and there's absolutely no consequence to them. The only consequence they understand is if programmers start to refuse jobs or tasks or quit based on violations of the code.
This is sometimes why I would favour certification or more responsibility placed on the programmer. Some programs should not be written or if they are written they should be held to a very high standard. We wouldn't have issues with high frequency trading if it cost a lot to provide insurance for that kind of development. Sure simple websites don't really need high standards, but as soon as you're past 1 million in revenue you're dealing with real money, real data and any bugs could have serious consequences.
On the other hand, I like being able to write little scripts that say "hey if something breaks, it ain't my fault".
It sounds like the main problem is with high fidelity O2 sensors at a cheap enough price point that these lookup tables can be banished. We legislated people carrying a block of platinum around in their car. Maybe $200 O2 sensors is the next step if the manufacturers can't be trusted.
If the Volkswagen case seems too clear cut for that, then think about PageRank, or Facebook's software that decides what to show to whom. Are we all so sure that every engineer who works with this code really knows, ex ante, all of the effects their changes might have down the line?
There are a lot of demands for criminal liability in this thread, but I'd suggest we proceed carefully. While it certainly looks suspicious, there are a lot of ways that weird behavior can creep into software that don't involve malicious intent.
Look at it this way: could Volkswagen engineers conceivably have written code that caused the system to fail all emissions tests all the time? (Yes.) Could they have done so without realizing that they'd made an error? (Sure.) Would we all assume that that bad code was deliberately introduced? I'm not so sure.
edit: See also a related discussion in the thread about the OPM data leak: https://news.ycombinator.com/item?id=10303950
===
* Or maybe some other, hypothetical cheating. Sure, in this particular case, maybe a particular software engineer or set of engineers knew exactly what they were doing when they wrote the code that enabled the cheating. But the thing I'm interested in is what happens when that's not the case, and how close that day is.
Modern cars have a lot of sensors and environmental inputs. A motor or emissions test then is quite abnormal in that some sensors will report standstill while you are going full throttle. The ECU needs to make sure it doesn't detect this as an abnormal system state and brake, shut the engine or otherwise perform corrective action that could put the test system or personnel in danger.
Apart from difficulties having manufacturers publish their code I think another problem will be proving that some (open source) software is actually loaded in some piece of hardware and nothing else or extra. Most products on the market can be taken, teared down and tested for as long as needed. With voting machines the time to proof some software was active during elections and nothing else is the time you have before you officially publish the results. This is an extremely short window and impossible for a large part of the public to verify.
This is an extremely hard problem to work around. They could let you dump the binaries of the software running on an individual car, and then you could compile the source code and compare the resulting binaries, but how do you know the car isn't feeding you a fake binary dump? It seems like a catch-22: I can't think of any way around this problem short of tearing the car apart, cutting all the chips open, and physically verifying them under electron microscopes.
On the other hand, if emissions testing would actually test what's coming out of the tailpipe under normal driving conditions, then that would seem pretty foolproof.
The way is to require reproducible builds. If the source is built with a specific compiler with known source and known parameters, and the source of the system in question is available, then running the compiler will produce the exact same binary every time if the compiler is sane. If all of this is published, then it is possible to check that the binary in the cars firmware and the binary produced by the published source match bit by bit. This isn't just some sort of academic exercise, but something that have been demonstrated to work in practice. In fact debian try to make this work for the whole distribution. This may not be totally practical yet, but this kind of checking can be done if the vendors are required to make reproducible builds and the regulators bother to check. All of this are big ifs, but at least in theory it is possible.
I believe that, for instance, the GPL demands the toolchain be replicable.
So under that constraint - really basic for any scientific purpose - the first test would be to run the proverbial 'make; make install' ...
Whatever extra layers of software verification we attempt to throw at the problem can be defeated by extra layers of deception from the manufacturer.
What if increasing government requirements are just not achievable? The laws are written by politicians, not engineers. If car companies are being asked to meet unreasonable performance metrics, what other choice do they have?
Cars have gotten a lot more efficient, I can get a 1.5L engine that puts out over 150hp. A 2L 4 cylinder engine can put out over 200hp!
These are huge savings compared to what used to just a decade ago, but they are still apparently not good enough to meet government requirements.
I understand that part of VW's cheating was a cost saving measure, but with all the talk about every car manufacturer doing it, one has to one, if 10 different people all independently come up with the same solution, maybe there is a problem?
Sure, and those are great for people who can afford the 10K-30K premium in price.
(Though the Prius does now have a lower priced option, one that may not be appropriate for many families)
All of the regulations CAN be met... but it is increasingly difficult to meet emissions, MPG, and Safety regulations, AND satisfy customers.
Safety equipment increases weight, which lowers MPG.
More efficient engines are generally smaller and less powerful, which does not satisfy customers.
In the VW case, reducing NO2 emissions lowered MPG due to the burn-off process.
One change to the fuel economy law that is sorely needed is changing the standard from MPG to Gallons per Hundred Miles. Differences between GPHM measurements are easier to understand [1].
1. http://billso.com/2008/06/21/gpm/ - See chart
They are also less safe for freeway driving.
On a recent vacation, I recently visited Albuquerque NM, driving around there requires a fast powerful engine to get up to speed ASAP, or accidents will happen.
In comparison, when I visited Boston, I rented a Fiat 500 and loved it, it was perfect for zipping around a those old narrow winding city roads, in and out of traffic, and finding parking. Heck I didn't pay for parking in Boston once while I was there!
Freeway driving is the norm for most of America though. A tiny engine won't due, it will however get someone ran over.
And if the testing mode never comes on during normal driving, well, that's not going to look very much like an accident, is it.
https://www.schneier.com/blog/archives/2005/08/new_windows_v...
I'm really not a fan of the classic British press discreditation technique of yelling "U-TURN" every time someone says something slightly different.
We shouldn't lose sight of who is really to blame for this problem. It's not the system administrators who didn't install the patch in time, or the firewall and IDS vendors whose products didn't catch the problem. It's the authors of the worm and its variants, eEye for publicizing the vulnerability, and especially Microsoft for selling a product with this security problem. You can argue that eEye did the right thing by publicizing this vulnerability, but I personally am getting a little tired of them adding weapons to hackers' arsenals. I support full disclosure and believe that it has done a lot to improve security, but eEye is going too far. As for Microsoft, you can argue that the marketplace won't pay for secure and reliable software, but the fact remains that this is a software problem. If software companies were held liable for systematic problems in its products, just like other industries (remember Firestone tires), we'd see a whole lot less of this kind of thing.
"The problem" here is "the Code Red worm".
One component of Schneier's recommendation is protecting researchers who practice full disclosure ("...attempt to muzzle security researchers who find problems"), but that's not really the core of his message in this post.
No one, I hope, thinks it's that simple. Businesses cannot be expected to put all their source code on github. Instead, this needs to follow the route that all other regulation goes. Instituting private but 3rd party review of the source code and testing, for which the manufacturer needs to pay a fee to support. I'm not saying this will catch 100% of the issues, but it's a lot better then what we have now and much more likely to work for businesses.
I imagine in the next few years more and more people will. Tesla have just announced that they will have self driving cars in 3 years.
This model could be applied to software as well. An independent organization like UL could evaluate the source code, talk to the engineers, and grant (or deny) certification. The source code would still be under NDA or appropriate privacy, but consumers and governments would have more to go on than just some company's word that they're not making anything dangerous or illegal.
Similarly, businnesses cannot be expected to document every dollar of income and expense. Wait. No. We can and do expect that.
Here's one of the latest occurrences:
http://www.theregister.co.uk/2015/05/06/antivirus_testers_st...
I'm skeptical of whether VW would have been caught any sooner, or would have changed their behavior, if they were forced to release source code; "and then analyze" is far easier said than done, especially with generated code (which is common in the automobile industry). I fear that if anything, forcing VW to release source code would have simply resulted in uselessly obfuscated "generated" code.
I'm skeptical of the proposition that taxpayers should take on the cost of analyzing reams of generated code without any context or documentation.
And finally, I'm skeptical that these calls for public access to source code are politically feasible, fair, or wise. The amout of intellectual capital that's spent on ECU design is absolutely massive. I don't see anyone in the tech industry calling on Congress to force Google or Microsoft to open source core components or reveal their software to regulators, even though vulnerabilities in their software could easily ruin or end lives.
It might make more sense to mandate that comapnies provide verifiable evidence that their safety-critical or regulation-relevant systems are properly designed, with a variety of avenues to compliance.
Releasing source code to the public and paying for at least one private analysis (to be selected by government regulators) would be one way of achieving this. This would probably be the easiest option for IoT companies (e.g. run-of-the-mill smart lightbulb manufacturers) whose source code doesn't contain any particularly valuable IP. And this would also force companies to pay up when they release hopelessly obfuscated code.
But this also opens the opportunity for other paths to compliance which, if designed properly, could address the safety concerns of the public as well as the fairness/property rights concerns of private entities. For example, one alternative path for companies whose IP concerns are legitimate could be use of formal methods. The regulation/safety specifications could be open to the public for criticism, and would be far more readable than a dump of generated code. And a few regualtors could double check that a trusted formal methods tool verifies that the specifications hold for the software running on the car, at minimal cost to both the car company or the general public.
But forcing Microsoft to reveal their crown jewels (they sell software) vs car companies reveal how an internal controller works (they sell cars), is not comparable in any way. Its disingenuous to imply that.
This makes two assumptions that I disagree with.
The first assumption is that software isn't a core component of cars. I think this is already not true. And to the extent that it is true, it won't be in 2-5 years. Software is at the core of emerging differentiations, such as self/assisted-driving features.
The second assumption is that that software doesn't or can't reveal sensitive information about (other) core components of cars. I think you'd be surprised at how much you can deduce about a physical system from its control software.
Finally, there's no clear bright line on which companies should or should not get to protect their IP. For Microsoft it's open-and-shut, but e.g. Google doesn't sell its search engine software. And what about IoT companies? "We're a software company that sells IoT appliances, not a lightbulb/car/robot/etc. company". So the only way to write such a regulation would be to write a regulation for the auto industry -- which is insufficient for the same reasons that Schneier talked about IoT and car companies in the same breath.
The controller we've been talking about control efficient engine operation, and conformance to existing federal standards. Not auto-driving cars (yet). There are powerful reasons to force them to be open, and only 2nd-order reasons to keep them secret.
In future I'd expect auto-driving cars would absolutely be completely open. There are even stronger reasons that emission-control issues, by far. Not running over kids for example. So we're likely to see a huge move in that direction.
As for a bright line, how about: if I can breath what you emit, or get run over by your software, then it belongs in the open domain? Pretty clear to me.
Just as car companies may just sell cars, Dell or Apple just sell computers. Why not provide the source code which comes with a new Dell/Apple laptop? In the end, how can a car maker, using Windows/Apple SW to build the SW for the cars, can trust them to not mess up? They are part of the tool chain for making cars.
For a car maker to provide source code, they would have to get permission from a lot of people. In this case, from Bosch. Bosch would require permission from their own OS supplier (probably Etas' RTA-OSEK/RTA-OS) and from other 3rd party suppliers. To be clear, these technologies are not directly available to the car maker. They never get complete source code from suppliers (unless some special agreement, but I doubt it). They can audit the SW/process though, which implies some level of access.
The engine ECU may not contain the new hype controls of self driving, but it contains a lot of know how, inventions, innovations. It's very nice to have a smooth running car, but it takes a lot of tech to make it run like this.
And no they don't have to reveal the source to the OS, that's a strawman.
If you tell companies they have to open source, you can be sure they'll be MISRA-C compliant. That doesn't mean their code is any less shit. Things like MISRA-C are useful in large part because where they are violated, you can be sure there was sloppy engineering. Giving people incentives to game standards whose claim to significance is largely correlative only guarantees that those standards become meaningless. In order for open sourcing to be useful, you need to analyze the code against some form of specification that concisely captures what that code is supposed to be doing, not just the absence of certain low-level errors or coding patterns highly indicative of certain low-level errors.
> And no they don't have to reveal the source to the OS, that's a strawman.
Why? A major bug in the OS would almost certainly make it impossible to provide any guarantees about the behavior of the ECU. And that OS is probably used in a lot of other safety-critical settings.
Sorry, but I still completely fail to see how your position isn't simply "primarily software companies deserve IP protections / primarily car companies don't".
> Disambiguating claims of company secrets, from the reality of coverups, is the whole point.
You're massively underestimating that amount of money car companies invest in both R&D and in engineering of software (Hell, car companies definitely spend more on CS R&D than the average software company, even if you restrict that to software companies working in safety-critical industries). This flippant attitude toward million/billlion dollar software investments based upon what companies happened to write that software is not going to fly outside of tech bubbles.
then I think they would know they were being watched and would act less horribly, at the very least. It's the same reason why you almost never see backdoors in free software. Just knowing that you could be watched makes it a lot harder to be sneaky, even if nobody is actually watching. Same principle as having placebo cameras that are actually non-functioning.
[0] http://www.welt.de/wirtschaft/article1708914/Ex-VW-Betriebsr...
What ever happened with that thing a few years back where some in the OpenBSD community were claiming the FBI was attempting to insert a backdoor?[1][2] I was always surprised with how little media attention that seemed to get.
1: http://www.linuxjournal.com/content/allegations-openbsd-back...