Why isn't all Internet traffic encrypted?
superuser.com
superuser.com
Fast forward to today and this legacy has remained. Many protocols are being retrofitted to support encryption, but it's similar to why railroad tracks are the width they are: Roman chariots used the same width.
Additionally, don't discount developer laziness. It's much easier to debug a plaintext stream than an encrypted one.
Only recently, with the spread of wireless networks, has the threat of en masse MITM and eavesdropping attacks become realistic. For a long time, it took serious knowledge to pull these things off on a wired LAN or within the walls of an ISP.
You could say the same about phone calls--why isn't the POTS network encrypted?--but that's because it's well constrained within wires run by trusted people and such cost is not justifiable; but with the advent of cell phones, protocols like GSM had encryption built into them, since telcos couldn't ignore that data sent over the air is readily captured and manipulated.
http://en.wikipedia.org/wiki/Wi-Fi_Protected_Setup
I think it's legitimate to want encryption even if you can't verify identity. Man in the middle may be trivial in that case, but it's still harder than just sniffing packets.
Encryption without identity validation is a worthy goal in itself, especially when we're talking consumer grade stuff.
There's this new feature of SSH key generators, where the user is shown an ASCII art of the generated SSH key, to be able to visually remember and recognize it. A standardized graphical representation of certificates could be used to confirm the identity of the router. Imagine walking into Starbucks or any WiFi enabled place, you see a sign on the wall that gives you the network name and the "VisiSign", you connect to the router, your computer shows you the VisiSign of the router, and you can visually confirm the identity of the router.
>why isn't the POTS network encrypted?
Because I can't pick up the receiver and listen to my neighbor's calls. That's exactly how ethernet doesn't work. I hear all, even on switched networks with a simple tool. Wireless? Even worse, it's the equivalent of using a non-switched hub. I'll even toss in that POTS should be encrypted point to point, but if you would have attempted it before Phil Zimmerman trail-blazed strong encryption for us commoners, you would most likely have ended up in a federal prison if you used non-trivial encryption.
So, to address the original question:
1. Encryption really was expensive in 1996 or so, but in the age of multicores, its questionable if there's a real cost now.
2. Performance hit. SSL handshake is long and annoying. We could use a replacement here.
3. Legacy. The hackers who slapped together networks and the various protocols we use weren't interested or couldn't properly secure this stuff. Thus the legacy of spam, plaintext everything, etc. Retro fitting security is hard. Its good if its there from the start like GSM.
4. Standards are hard to change. Imagine telling all the browser makers and open source projects that we should all switch to something that won't add to the bottom lines/download rate/market penetration, but instead cost them time and complexity as well as add server load. Google can't get anyone to care about SPDY. Who is going to re-architect this stuff? We're essentially living in a world convcieved in the 70s, implemented in the 80s, and made popular in the 90s.
The easy answer is to stop allowing non-encrypted wifi. I'd love to see the wifi consortium just default to a random key for non-authenticated guests in whatever the newest version of wifi will be. It'll be transparent to the end user and probably stop all these local attacks. It won't help you past the gateway, but right now between the client and the gateway is the problem area. Heck, why not make all networking gear do this, even on wired networks? We have the processing power.
A lot of "budget-priced" hosting services are near-free for http and heavily-surcharged for https. - Pricing it to make it seem as if there were some huge cost difference. Makes a difference when shoestring clubs/nonprofits want to throw up a cheap static site.
Turns out that you can only have one SSL applied to one IP because browsers* can't tell the webserver hosting 1,000 sites which particular site you're requesting, because of, you guessed it, encryption.
Essentially, you're paying for a dedicated IP not CPU.
*theres a fix for this supported by several browsers, but it will never be backported to legacy browsers and the millions who won't upgrade for many years so its unsafe to assume you can use this method, thus one IP per SSL for the foreseeable future.
The problem with that is that systems like Skype require strong encryption to work at all, and Skype manages to provide high quality voice transmission despite using encryption on a wide variety of OSs and hardware.
It will probably be more technically difficult to make Skype friendly to snooping than it was to make it secure.
The two parts are identity and obfuscation.
Obfuscation protects you from listeners who do not control the network, and identity protects you from listeners who do control the network.
Together, you know only your intended recipient is getting your message, but apart, at least you are protected from certain threats.
So even if we just had obfuscation, that would protect us from most threats. When people say "we need encryption everywhere", I think generally what they mean is "we need obfuscation everywhere" to protect from the worst threats, which are those of the uncontrolling listener.
The flip side is that the uninformed will get complacent and assume that obfuscation is giving them identity protection as well.
There's really no such thing as an "purely passive" attacker. Since the 1990s, probably starting with the deployment of switched ethernet, active attacks have gotten steadily easier and more popular, and passive attacks less so.
The frustrating thing here is, we are so close to having this problem licked, but so many smart people are spending time concentrating on problems we don't have. What we need to do is to come up with a credible replacement for the Mozilla/Microsoft/Google-controlled CA system. It's probably no harder to solve that problem than it is to push adoption of weak unauthenticated protocols.
Of course, there are other problems preventing small and large sites from using SSL, like the requirement of a dedicated IP address (which SNI fixed, but is again not supported in all popular browsers) and the overhead.
Why technically
Why politically
The technical issues are real and deep but they can, were, and continue to be tackled. The bigger issue is that successfully addressing and deploying robust encryption solutions is not a political priority (quite the opposite).
Keep well in mind that for many online in the 90s, the assumption was that eventually standards for email encryption, as one example, would become well deployed. The fact that email is today routinely not encrypted (disregard ssl connections for the moment, a whole different barrel of fish) is a demonstration that interested power structures (governments, US primarily) were successful in limiting uptake.
There are real challenges to successful distributed crypto, but the political forces in play had the effect of early work withering on the vine for the most part.
Nobody wants the entire Internet encrypted more than the giant banks that Redditors believe control the political system.
What malign forces do you believe the government is actually wielding to retard adoption of crypto? Who controls how much of the Internet is encrypted? Because the Internet is an end-to-end system design, the two parties most responsible for crypto adoption are your browser vendor (which vigorously supports crypto adoption), and the people who run servers (most of whom vigorously support crypto adoption).
The onus is on you to provide evidence to back your silly argument up.
Outside of payments, where the government and industry have pushed for strong integrity and authentication (although not really confidentiality), government and big corps have hindered the deployment of crypto.
There has been some improvement in the past decade or so, at least in other regulated areas, both in regulation and in industry self-regulation/compliance standards.
I think the majority of the reason cryptography isn't more widely deployed is that it's 1) hard to do well and 2) most people writing applications have a hard enough time getting a non-encrypted form working and 3) few people make it a requirement as a customer. However, government has definitely hindered ubiquitous crypto deployment.
But that was the 1990s. The government does not in 2011 believe you are shipping "munitions" when you allow open downoads of software that incidentally encrypts traffic. People do it all the time now. And, to be fair to the government: nobody saw the mainstream Internet coming, and prior to that, crypto basically was a munition.
I agree with your (1) (2) and (3) reasons. I just don't see anything the government is doing today, or in the last 10 years or so, to hold back an encrypted Internet. The people I know in government who think about this stuff would dearly like to see a more secure Internet.
Indeed, Yahoo, Hotmail, and Gmail could get the ball rolling here completely transparently to users and with relative ease, thereby preventing thousands of phishing attacks in the process. If they did, then webmail would suddenly become useful for talking to banks, stockbrokers, foreign dissidents, avoiding craigslist scams, etc. etc.. But not only have Gmail et al not implemented any features like these, but they've not even uttered a peep as to why.
Their silence here is conspicuous.
It doesn't help that Gmail, a web app, has no secure was of implementing PGP with people's "real" public keys.
Apple bakes encrypted email into Mail.app. It has an almost transparent UX. Nobody uses it.
Demand is what's holding back encrypted email. Nobody cares about it.
Look at Ubuntu: they're constantly improving the user experience of encrypted home directories. In the latest release, this is as simple as checking a box during account creation (it creates the keys under the hood and uses your login password as the passphrase for the data key etc).
Also re: gmail has "no secure way". I read the white paper written by your company on the problems with encryption and javascript. AFAIK, if all traffic is exchanged only over SSL, I don't see any reason why encryption can't be done in Gmail in javascript (unless you're going invoke quality of the random number generators of browsers).
Here's an UX I can think of: (1) generate key pairs in the background in the browser for all gmail users. (2) transparently use this to encrypt to users within gmail.
Start from there. Then add the ability to import keys of contacts.
What aspect of the UX I described do you think is confusing? In the step 0 that I proposed (encryption only within gmail), there is virtually NO UX.
Wrt "nobody is asking for it". Then why is ubuntu/windows adding the feature to encrypt dirs? Btw, at a company I worked, they were rolling out SSL certificates to sign email. So I don't agree that "nobody is asking for it".
Wrt your point about the comparison being "unfair": I don't think anybody disputes that the "usual" way of doing public-key encryption (rigs of trust etc) is too complex. At the same time, I remember the past when encrypting data dirs was an immense chore. If that UX could be improved, why not the email encryption UX?
Or click the "encrypt" box (which causes Gmail to determine whether it knows the public key of every one of an email's recipients).
Or click the "sign" box, to add her signature to the email before it sends?
Sure, sometimes Gmail would have to say "this email cannot be encrypted because of insufficient information about recipient X". But if anything this might lead to more people using large email providers.
Sure, consumers don't know that they want PGP. But when they suddenly discover that can trust that all their messages are seen by only a particular recipient, and that they'll always be able to tell when a message is or isn't from their coworker, they'll quickly start insisting on it the added security.
Virtually every intra-company mail I send or receive is PGP-encrypted (we're getting to be a fairly big company by HN standards, too). I assure you: my mom could not deal with PGP's (constantly evident) corner cases. How could I possibly think it's a credible solution for the whole world?
What are the challenges with using public-key encryption with a changing group of people? Key exchange? Remember we're in a more-centralized, always-connected world now wrt email than we were in 1995 (remember offline email writing?). The problem should be simpler now if anything.
People have false notions about their email and web traffic in general. The closest previous analogy is shipping packages and letters. There're laws preventing USPS from snooping. People seem to reasonably assuming that that's the case with electronic letters too.
And nobody in the mainstream is disabusing of this notion (encryption is mostly shown as being used by terrorists and swindlers). When they become aware of how naked their communications really are, they'll learn how to cope with encryption (including your mom :)).
I'm imagining that the webmail providers could implement it piecemeal, and though I'm speculating here, I think this slightly-less-secure-than-complete-PGP approach would address a lot of the problems you're referring to (I'd be eager to hear why you disagree, though!). Corner cases should only be a problem if the approach is all-or-none, right? And if done properly, a piecemeal PGP implementation would still be more secure than no implementation at all.
From the user's/your-mom's perspective:
a) conversation threads would be kept separate from each other depending on whether or not they were secure
b) some messages would be signed, explaining what is known about the sender and why (e.g. "this message has been verified as originating from xx@gg.com", or "message is known to have come from John Smith at 2nd National Bank, with email address xx@gg.com", or even "John Smith, with the following verified profile".
c) some messages would be encrypted, telling the user "this message was securely sent to you, and person X, and person Y, by xx@gg.com."
d) when a user sends a message it would default to the most secure mechanism that all of the conversation participants allow, given what it knows about the recipient
Even if Gmail was the only provider that implemented this, especially given two-factor authentication, users would immediately get secure conversations at least in conversations that included only other Gmail users, which would go a long way to dealing with the fact that it's the lack of other people using PGP that makes adopting PGP kind of pointless.
Given that Google provides webmail to many universities and businesses, implementing PGP would immediately turn on secure and verified communication within those organizations: the benefits of PGP there would be instant and enormous: identity verification, timestamps, and signatures are the only reasons people still shuffle paper around. No more clumsy and expensive university-stamped transcripts; no more running around trying to find people to get their signature; no more running signed documents between various offices, etc.
Furthermore, your mom might already use Facebook (or another social website) and if so is already experiencing the benefits of verified identity and higher-trust communication. On fb you have absolute confidence that you're actually posting to friend X's wall; and when you're messaged by friend X you're absolutely sure it's X (unless someone gained access to X's account). This implies that similar such UI-based cues could be used for webmail, no?
Indeed, one could characterize part of the success of facebook (and other social networks) in terms of identity verification: you know that information posted by person X has actually been posted by person X (vulnerabilities notwithstanding); you know that what you post will not be seen by people whom you don't want to see it; you know when person X posts on your wall that it was actually the X-that-you-know and not some other one or a fraud; you know by virtue of X's existing relationships to your other friends that it's the X-you-know rather than someone else.
Ultimately, I think Google hasn't implemented PGP, in spite of all the incredible benefits, because it doesn't want to encourage its users to be more private. Is this evil?
tl;dr partial PGP implementation on webmail would avoid corner case problems while providing huge benefits to millions of people and organizations
I think it's actually a chicken-and-egg problem. The people who care a lot about it can't have it because they need to communicate with everyone else, the people who care a little don't want to bother with using encryption for some contacts and no encryption for others, and the people who don't know and/or don't care don't get any of the benefits.
I think two things would make a big difference: 1. marketing end-to-end e-mail encryption as an anti-spam measure, and 2. someone who cares a lot about it implementing an end-to-end, non-government-controllable system that's undeniably easy to use, then releases it for free and gets all the major e-mail platforms to adopt it (obviously much easier said than done).
Why is so much bank security nothing more than ridiculous security theatre? I agree they have the money, they have the need, they have enough smart programmers; so why do they fail so hard at security?
2) Selective perception. We only notice when they fail. On the whole maybe they do do a pretty good job and nobody notices when things aren't wrong.
3) Sometimes it's an insider job.
4) Dumb users; no amount of company security can stop a successful phish or keylogger or whatever.
5) Cost-benefit. Sometimes it really isn't worthwhile to pour as much security as you possibly can at a problem. What if a bank guarded every transaction by having a live operator call the customer at a pre-known number? Sure that's pretty stiff security, and completely impractical in a competitive capitalist marketplace. That's an extreme example but it goes to prove the point. Sometimes it's cheaper just to clean up aftewards.
Security is a thorny problem; you can't always just magically have more security by throwing more money at it.
Excerpt from a 1998 essay, "Security Pitfalls in Cryptography" that sums it all up:
"Strong cryptography is very powerful when it is done right, but it is not a panacea. Focusing on the cryptographic algorithms while ignoring other aspects of security is like defending your house not by building a fence around it, but by putting an immense stake into the ground and hoping that the adversary runs right into it. Smart attackers will just go around the algorithms."
http://www.freeswan.org/ending_letter.html (tldr: nobody cared)