Class Breaks
schneier.com
schneier.com
Sure, it's easier for a single person to temper with a single paper ballot or even poll box, but it is much harder on a large scale. Tampering with every physical boxes and tallies in a country requires much more effort and organisation than exploiting a single vulnerability that applies to an endless series of identical machines.
Similarly, propaganda is also more effective.
(Though, I will confess I have not tabulated this, so would be very curious in full numbers.)
There is probably a lot more subtle election fraud that is never detected.
Now, overt manipulation of the voting populace through propaganda or other means is very common and has definitely happened. Ballet fraud, though? Are there any documented cases of that being a thing that mattered?
If there's one lesson that we can take from all the data breaches from top companies and high-profile government officials is that we are terrible at securing software. The 21st century Ted Kaczyinski isn't going to need the post office; he's got a MBP and SSH.
Yes but think of the probabilities that have to happen are:
a) Knowledgeable person
b) Motivated person
c) Evil or mentally unstable person
d) Determination and ability to overcome adversity.
And the person or person who has to decide that Tesla is what they want to exploit instead of another car manufacturer or another object and so on.
Compare this with someone who simply wants to take their car and drive it into a crowd.
If we're assuming hackers that realize they can lock down a person's car in exchange for ransom... Simply look at how many hackers do it already.
If we're assuming a broad attack by a foreign entity... Simply look at how many hackers do it already.
You have to multiply the probability by the harm. Yes, most people will just drive their car into a crowd, killing - let's say - 10 people.
But the man who perseveres and drives EVERY controllable car into a crowd can kill thousands, if not millions.
Anyone of us could drive into a crowd and kill 10 or many more people. Yet it happens extremely infrequently, compared to at least how many times it could happen.
1. Computer systems that run important things (power, water distribution, logistics, financial networks, communication networks, military systems, air traffic control, individual plane/ship control, etc) are inter-networked and vulnerable.
2. The world is full of evil people that hate the West and want to see it suffer.
3. Nothing has happened.
How can all of these three things simultaneously be true?
1. Western countries are full of soft, populated targets with minimal protection.
2. The world is full of evil people that hate the West and want to see it suffer.
3. Very little has happened.
It's not quite as strong, since 3 is a weaker claim, but the attacks we do see are pretty rare relative to what you'd expect.
In both cases, I think the answer is that #2 is wrong. A lot fewer people are out to get us than we're led to believe.
> It's a world where driverless cars are much safer than people-driven cars, until suddenly they're not.
But this runs counter to the "don't roll your own crypto" philosophy. Crypto in this context can be any sort of important aspect of your device that you want to secure from hackers.
Right now it's a bad idea to do things on your own, but as systems get even larger and interconnected it may be your only defense.
Fevers are often created by the body to "burn" the offending pathogen from functioning properly and infecting further. Diarrhea is an attempt to purge the body of the disease from the areas it thrives in most.
So in this case, roll your own crypto and use known strong crypto. Don't rely on either on their own.
Example: use end-to-end encryption apps like Signal, but if necessary, communicate in coded language that only the person on the other end would understand.
It's a clear case for technological diversity. The hard problem is that economies of scale work against technological diversity. It's similar to what happens with monoculture in biology.
There really isn't a technological solution for class breaks. Go big and you replace one kind of class break with another (a top-tier security system for hotel card readers just focuses the attacks on that system). Go small and it isn't cost effective or plausible at scale (everybody install gentoo! Compile your own kernels!).
Honestly, I'm stumped. We rely on these systems more and more to squeeze out every bit of productivity we can get, but that only leaves us at the mercy of our systems.
Is it even possible to fix this kind of situation before it really turns on us, or are we doomed to suffer a cataclysm we cannot yet imagine?
The former will be phased in as countries legislate it. Legislators won't be concerned with the slippery slope between abacus and iphone, the courts will likely draw a distinction somewhere around anything north of "silicon-based electronics with the right resources for an IP stack". A few countries will be bastions of Free Computing, but many/most will draw up similar laws.
The latter will be held back by Metcalfe's Law until some near-global momentum can be built. It would likely take a simultaneous attack on many global corporations/political parties/beloved individuals, but it seems like that's just a matter of time. Faced with repeated attacks, the public won't accept "but we designed it that way intentionally" as valid.
However, software engineering will change into a much more regulated environment. Writing software for the web will eventually become like writing software for the space shuttle: slow, rigorous and very expensive.
It will also kill the hacker culture, as hacking together a piece of software will become like hacking together a bridge: illegal.
I for one would prefer most vital software to be developed slowly and methodically. We insist on strict standards for our bridges and roads - why don't we do the same for our information infrastructure?
Move fast and break things can break people too.
But before we celebrate its demise: the side effect will also be the same ones seen in the civil engineering world: creative ideas take decades to come into existence. As you mentioned in your other comment, there was a time when railroad building was exciting, and in that time, people died doing exciting engineering. Civil engineering tools are now orders of magnitude better than they were years ago, but there hasn't been an explosion in creative structures, even at the lowest level. I also think many of the perks (ie: salary) of the software engineering industry are intimately related to the 'move fast and break things' culture. When that leaves the industry, it may be less of a good riddance than you think.
This may be my perspective as an outsider though, I don't work in software. But I have considered making the jump to software engineering, because 'moving fast and break things' sounds fun.
Fundamentally we agree, but I'm a pessimist.
It's easy to grumble about regulations until their reasons are forgotten. Then they get repealed and the problems come back. The mortgage crisis is a prime example.
I don't think this is such a bad future for software either. Much of my job writing software for higher education involves adhering to complex policies. These policies are necessary to remain FERPA/HIPAA compliant, which are necessary for their own reasons. Playing fast and loose is taking out a debt for an uncertain future.
If we have to prove _everything_ on your computer, from your calculator or Pokemon, how much do you think a license of windows will cost? $250,000?
Linux/FreeBSD/OpenBSD/Minix will be dead (no one to sponsor certification and then put it in the public).
Crazy "best-practices" (change passwords every couple days, password must include symbols, letters, numbers, upper-case and lower-case in a random order, but be no shorter or longer than eight characters)
Must have a full team of lawyers to prove that everything done fit the letter of the law, and that any hacks are not your responsibility
"Shinyness" is what allows you to have free VSCode and Atom.
Sure, I miss the 70s when you could get a text editor measured in bytes (but, btw, electron is probably more secure than 70s unix) but I definitely like the cost/convenience of modern software.
[1] http://www.dopl.utah.gov/licensing/commercial_interior_desig...
[2] http://www.dopl.utah.gov/licensing/cosmetology_barbering.htm...
... risking anyway to end up in the "B Ark" ...
http://www.geoffwilkins.net/fragments/Adams.htm
(obligatory reference to Douglas Adams, nail technicians and interior designers are not so different from phone sanitizers and hairdressers)
Depends on where you buy your land, most likely. If you buy in an unincorporated territory, you might get away with whatever if there are no county or state regulations. If you're buying in a city/town/village, you can expect that basically any construction project will be subject to code requirements and permitting. To get permitted in such a situation would typically require an appropriately engineered design with signoff from a civil PE.
I wouldn't be totally pessimistic like this. Increases in regulation and quality standards will happen along with increases in programmer productivity as tools further improve. We're still living in the stone age of programming languages and system infrastructure.
See: aviation (ICAO, FAA), automobiles (WP.29 [1], NHTSA), wireless transmissions (ITU, FRC/FCC)
[1] https://en.wikipedia.org/wiki/World_Forum_for_Harmonization_...
Or, it could bifurcate. Look at aviation. If you want to fly an airplane in the US, it needs to be something that has been approved by the government down to each nut, bolt and rivet. All onboard systems (mechanical, electrical, software) and every vendor providing them have undergone expensive and rigorous government certification, and you have documents showing it. You can't so much as change out a switch in the cockpit without going through an official approved service provider! But there's a totally separate "experimental" category carved out for people who want to build their own airplanes, where the rules are much more lax and there's far less oversight. As long as you keep good records and abide be some basic rules, you can pretty much build and fly whatever the hell you want. The major operating limitation is that you can't fly one for hire or carry people for compensation. By some great miracle, these separate, parallel tracks, certificated and experimental, co-exist successfully and planes aren't falling from the sky.
It may end up this way for computing: A "serious software" regime for most consumer- and business-purchased products, subject to slow, rigorous, expensive regulation, and a parallel "hobbyist" track for at-home, non-commercial hacking and making where anything goes.
So, we do have some technical solutions that could greatly reduce the risk. I think the main barriers preventing these techniques from being used as widely as they should right now are that 1) secure and insecure products are basically indistinguishable to consumers, so there is little economic incentive on the part of product manufacturers to ensure their products are secure, 2) we have an entrenched programmer culture (especially in systems programming) that thinks the solution is to "be more careful" and "don't hire dumb programmers", and 3) we have a whole lot of really complicated C and C++ programs that work really well and people are happy with them whereas if you want to ship a product with, say, all of the code from the OS up written in Rust, you're sort of blazing your own trail and you'll have to write a lot of it yourself.