Hacker News Reader

> However, it doesn't address intercepting and recording communications in-transit, which is extremely commonplace these days.

MTA<->MTA SMTP transmission is increasingly SSL these days, and end-user MUA<->MTA transmission has pretty much been entirely SSL for a long while. Google cites about 80-90% of its MTA<->MTA transmission is encrypted (https://www.google.com/transparencyreport/saferemail/).

Yes, see my link to the BULLRUN program. There were quite a few follow-ups as well.

[1] https://freedom-to-tinker.com/2015/10/14/how-is-nsa-breaking...

[2] https://arstechnica.com/security/2015/10/how-the-nsa-can-bre...

[3] https://blog.cryptographyengineering.com/2013/12/03/how-does...

And so on and so forth.

Also keep in mind that they are retaining a full take online-- storing all traffic on the internet. Even if it can't be decrypted in realtime, they can come back to it in, say 10 years, with all the advancements in brute forcing and potentially quantum computing available in 2027, to take a look at your 2017 activity. The emails you wrote, the websites you visited, all your chats, all your searches, saved forever.

There is no evidence indicating that anyone can break encryption for mail servers using STARTTLS.

Not right now, no. But if you start saving encrypted streams, and then come back to them 10 years later when, perhaps, a TLS vulnerability is found...

I don't know what the media coverage stated, but the source material indicates that full-take is only stored in a 5 day rolling buffer. I believe 30-day for certain flagged selectors. Not at all indefinite, that is not possible.

As of 2011, GCHQ stored their full-take for 3 days and metadata for 30 days. Do you believe they haven't increased capacity in the past 6 years?

Again, the discussion comes down to evidence versus reasoned speculation. Evidence of intelligence agency activity is generally pretty tough to come by.

I'd also believe the country's data transmission volume has increased in the same timeframe, and given how IOT is rapidly scaling up, probably a lot faster than storage capacity.

They have techniques to reduce the data corpus, called "massive volume reduction" or MVR in the GCHQ "Mastering the Internet" rather pompously named leak. It discards junk traffic and assumedly does deduplication, etc. Remember, Tempora went online in 2011, 6 years ago. Techniques have certainly been refined since.

[1] https://www.theguardian.com/uk/2013/jun/21/gchq-cables-secre...

[2] https://www.theguardian.com/uk/2013/jun/21/gchq-mastering-th...

But we're talking encrypted streams here. No algorithm, real or imagined, can dedupe or compress random noise.

Additional data intake would greatly increase costs as well. I guess I'm not fully understanding, is the implication that getting more data would lead to more useful information?

From reading the source documentation itself (not editorialized coverage), it seems they are already swimming in almost too much data. Their focus is much more likely to be aimed at data refinement at this point, as adding even more data would make it far more difficult for analysts to do their job.

It absolutely will NOT be more useful! That's one of the main takeaways from Snowden's leaks-- mass surveillance does not make us safer. Thing is, the three-letter agencies strongly disagree.

It is unlikely that many servers are strong enough to resist penetration and theft of the mail server's private key. That key then can be indexed and matched as well. Is your mail server refusing downgrades to non-PFS ciphers? Is your kernel not affected by zero-days leaking the state of your secure random pool? Are there vulnerabilities in openssl we don't know about but collection agencies do? Is the reason that the government hasn't announced the CNSA Suite yet because the NSA is busy taking advantage of EC weaknesses combined with quantum computing capability? To say nothing of various direct TLS attacks, which are easy enough to translate from HTTP to SMTP, are agencies putting these to work? How far does the integration of passive collection and active attack go these days - there's plenty of evidence to indicate they have been significantly combined in the past.

To say there is no evidence that anyone can break a TLS session is a bit like denying climate change.

> To say there is no evidence that anyone can break a TLS session is a bit like denying climate change.

None of what you have stated is evidence, and nothing from the leaked Snowden material is evidence. I did not say that they cannot do it as I have no way of knowing that, only said there is no actual evidence at all, so this devolves into "what-ifs" which lead to helplessness. I do not see how that is good for anyone.

Do MTAs check the certs of each other now? I was under the impression that they largely didn't.

They either don't or will happily silently downgrade to SMTP even if previously they've connected via SSL. So an active interceptor can still intercept communications.

By and large the only way to prevent man in the middle and downgrade attacks is to use DNSSEC and DANE. Without that, an attacker can just filter out the STARTTLS option in the SMTP EHLO response and the mail will be sent in plaintext.

However, active attacks like downgrade attacks and TLS man in the middle are not popular for large scale surveillance. The reason is that it is easy to detect.

Or you could just use the existing standard TLS handshake instead of STARTTLS to upgrade the connection. DANE for this would be a step backwards, as you would just give governments the control of the PKI system.

Edit: There is, however, something you could do with DNSCRYPT (or blockchain), that goes beyond just in-transit encryption. If you could publish your public key to a trust-worthy registry then we could have transparent body encryption. You could move email storage locally (with encrypted cloud backup), but still have DDoS resistant MXs.

For large scale monitoring of traffic, active attacks are very unattractive. If a government would manipulate dns on a large scale, that would cause a lot of trouble.

For targeted attacks, using PGP is the way to go. Though it comes with the price of manual key management.

It would be nice if somebody can do decentralised authentication at scale. But I haven't seen it in practice and I doubt it is going to happen at all.

PGP works on individual message level. If email encryption is ever to get ubiquitous it needs a more comprehensive approach. It should start with a local email server that keeps all messages in an encrypted vault and can communicate in a secure manner with other clients that implement the “secure delivery” protocol (which would be indicated by being able to securely resolve the public key of target email address, and from an UI perspective by a special “Secure Send” send button).

Manual key management will never succeed. Key discovery needs to be aggressively opportunistic yet secure. It’s the only way to go.

Disclaimer: I'm only a consumer of cryptography, not a cryptographer.

I have the feeling that secure email needs to be completely separate from SMTP; clients might be able to use both, in the same way that Signal will also use SMS when necessary. It should probably use a double-ratchet-based protocol, but cannot rely on centralization for contacts discovery and key exchange (because then it wouldn't be email-like).

Among things that currently exist, it seems like the Matrix protocol[0] and OMEMO[1] seem basically suitable as a transport (both of them use the Olm protocol developed for Matrix, I believe).

[0]: https://www.matrix.org/ [1]: https://xmpp.org/extensions/xep-0384.html

You can absolutely replace SMTP but whatever you replace it with needs to be at least compatible with it. Lots of warnings, Insecure Email Ahead, but it needs to be compatible.

I also don’t think email should be e2e by default. Next gen email will never get adopted if Gmail can’t index it. What you want is to give them the option of selling “Secure Cloud Email Vault.” Something that’s encrypted or partially encrypted (just body) on their servers, like Amazon is selling encryption for S3.

But the protocol can also support fully local encrypted email (as detailed in my previous) for those that want that.

Edit: Oh also for the actual cryptography there needs to be a way to give someone (like Court Order) decryption key to a single message. This avoids a lot of issues. Governments are not just going to let you deploy fully encrypted email by default.

> I also don’t think email should be e2e by default. Next gen email will never get adopted if Gmail can’t index it.

Fallback is fine; in the age of "surveillance capitalism" security for the moderately interested is the best we can hope for.

> Governments are not just going to let you deploy fully encrypted email by default.

What do you base this on? What would they do?

> What do you base this on? What would they do?

PATRIOT Act Vol. 2. It’s the lizard people I’m worried about.

I'm very curious if you know a decentralised authentication mechanism that scales and works for e-mail. I don't know any.

Without, improving over PGP is mostly wishful thinking.

As I said DNSCRYPT or Blockchains like Etherium/Namecoin. PGP’s WoT model is a complete failure.

How will using SSL/TLS protect you from the NSA?

They must surely have the ability to generate valid certificates.

I don't get why this was downvoted. The chances are that the NSA has obtained at least some certificate authorities' root certificates must surely be pretty high?

I not why not? (I may be misunderstanding something about root certificates).

It's actually even simpler than that: self-signed certs are rampant among mailservers, so it'd be trivial (comparatively speaking) to perform a MITM attack. Hopefully we'll eventually get to the point where everyone is using Let's Encrypt or has otherwise ditched self-signed certs, but until then, the likes of the NSA wouldn't even need to bother with the CAs to snoop on mail traffic.

Because using a CA to issue a cert for you improperly is a good way to burn your CA. So it'll probably be used very carefully and in very limited scope because it's detectable and leaves undisputable proof.

For SMTP it's different since people aren't watching as closely. -- but then the CA angle is irrelevant because there's no verification. Though I'd be surprised if Google doesn't have some sort of program to look at these things and try to detect MITM or just gather info in general.

Didn't they just add a little security indicator to Gmail? Google could probably single handedly force TLS verification on email by saying it'll increase deliverabiliy.

The dirty secret nobody wants to talk about is that MTA's don't do certificate validation. You can't or you'd never deliver email successfully because of the number of self signed certs out there.

MITM on SSL/TLS SMTP is child's play

Hopefully Let's Encrypt (and ACME in general) and similar efforts will help to alleviate that issue.

Let's Encrypt in particular seems to be more geared toward websites, but I actually did manage to put it to use rather trivially on my mailserver (which previously used self-signed certs) thanks to 'acme-client' on OpenBSD.

Postfix and Exim both support DANE/TLSA. When I email somebody with TLSA set up, or they email me, the certificates are validated using DNSSEC signed DNS records:

  mike@snake:~$ dig +short mx grepular.com
  20 mail.grepular.com.
  10 mx1.grepular.com.
  mike@snake:~$ dig +short tlsa _25._tcp.mx1.grepular.com
  2 1 1 60B87575447DCBA2A36B7D11AC09FB24A9DB406FEE12D2CC90180517 616E8A18
  2 1 1 B111DD8A1C2091A89BD4FD60C57F0716CCE50FEEFF8137CDBEE0326E 02CF362B
  mike@snake:~$ dig +short tlsa _25._tcp.mail.grepular.com
  2 1 1 B111DD8A1C2091A89BD4FD60C57F0716CCE50FEEFF8137CDBEE0326E 02CF362B
  2 1 1 60B87575447DCBA2A36B7D11AC09FB24A9DB406FEE12D2CC90180517 616E8A18
  mike@snake:~$

That assumes that nobody can MITM either your DNS server, the your ISPs DNS cache (if you use it), or Google's DNS cache (if you use it).

Or it assumes DNSSEC.

It does assume DNSSEC yes. Which is a requirement of DANE.

Who are you emailing? Gmail/yahoo/other technically competent organizations?

Try working with small businesses, lawyers, etc who self-host. It will fail and clients will scream bloody murder unless you turn off validation so their emails can get delivered.

> It's very reasonable to assume that everything, EVERYTHING, you do on the internet is recorded and stored by the government. This bill does not address that.

I mean, that's hard to believe. I work for a large storage vendor. If this was true, the feds would be our largest customer by far. They aren't. I don't think SE Asia makes enough disk to store everything indefinitely, and given exponential network traffic growth, even storing the most recent few years is a problem.

I might consider that they're just not buying it from your company, or you aren't privy to the transactions to those gov't customers. How much storage does 1.5 billion buy? (Ok it's not all just storage...).

https://en.wikipedia.org/wiki/Utah_Data_Center

The explosion of data you mention is a real problem; though thinking about it, one might manage with a mix of light filtering, leaning on organizations that already store the relevant data, etc.

That is just one opinion against another. With little fact to support any side.

In your comment, for example, you seem to suppose that those agencies do not buy storage on secondary markets. Can you confirm that? Because if they do, then your sales statistics are almost meaningless in this regard.

Also, it seems like you assume that everything the agencies collect is stored "as it was transferred" (i.e. if the same YouTube video was viewed millions of times, they would store it in millions of copies). I think it is safe to assume that these agencies are a little bit smarter than that and if they store everything, they have some kind of compression in place to minimize redundant storage of the same chunks of data.

In any way, I think that until someone like Snowden speaks up again and brings evidence from inside of these agencies, we are only left with speculation.

There's no speculation on the UK. We know the GCHQ does this. Of course they generate quite a bit less traffic than the USA, given their smaller population.

We don't know whether the NSA does "Full Take" today. That's speculation. I would be astonished if they didn't, and it is reasonable to behave as if they do.

It's possible, it was possible 4 years ago, and storage is not getting _more_ expensive over time.

Exclude all the audio and video data streamed over the internet, as well as the contents of torrent traffic and other data useless to the panopticon (just keep the metadata), and I'd imagine the amount of data you'd be left with would be very possible for the US government to store.

> This bill is great in that email stored on your ISP's (or Google's) servers is treated identically to that on your computer at home-- they need to get a warrant. That's a great step forward. It isn't BS.

Except for one major difference: Gag orders. When a warrant is served to Google to access your email, you may never know about it. If they show up at your home and take your email server, who you gonna gag?

Unreasonable or overly-broad gag orders are one reason I always advocate running your own services whenever possible (granted, this needs to be easier).

The distinction you mention is real, although not all warrants against providers come with gag orders, and some warrants against people's homes authorize surreptitious searches, such as by breaking into the home when the target is away. (This was done for a personal computer as early as 1999 in the Scarfo case.)

Lately law enforcement has been seeking warrants to hack people's computers and some of them have been granted; typically those have also been surreptitious searches. I think this is the greatest risk for surreptitious government access to self-hosted information, because some agencies have been getting pretty excited about this and there are a lot of contractors who will supply them with vulnerabilities and tools even if they don't have the expertise to develop those themselves.

That is certainly true. The difference is that before they didn't need a warrant at all if your cloud-stored data was over 180 days old. A court order alone was sufficient.

> That's why it's absolutely critical that everything you do is end-to-end encrypted. These government agencies have the resources to crack most encryption, too [3] but at least you aren't making it trivial.

This comment in particular seems to imply that government agencies can break e2e encryption. However, other sources would suggest that's not entirely the case. Snowden, for example, still seems to believe PGP works. I don't have reason to believe that it doesn't, or that the NSA can currently crack any arbitrary 4096-bit RSA keys.

Further, there's a lot to be said about other modern encryption standards (ed25519 curves, forward secrecy and asymmetric double-ratchet protocols like Signal), which I don't believe have been shown to be compromised. I understand the importance of e2e encryption, but it seems that you're overstating the governments ability to crack it. I agree that SSL and TLS don't have the best history, and maybe we shouldn't rely on them, but it is also important to know that there are solutions out there which we still have quite a bit of reason to believe we can trust.

> This comment in particular seems to imply that government agencies can break e2e encryption.

Much E2E encryption can be broken by government agencies. For HTTPS for instance, the government is a certificate authority accepted by your browser - as well as there having been multiple cases of collaboration with well known CAs to have certs minted for them. In this case there's no need to break RSA. In any case, much of TLS is open to various kinds of breakage (that is not as hard as RSA), and NSA docs showed a number of instances where intelligence agencies would sit behind load balancers and at data center chokepoints in domestic companies to gather information where PKI no longer poses a problem (behind the load balancers being referred to here as "end-to-end").

Basically, there's many ways to thwart "e2e" that isn't cracking 4096 bit RSA. Intelligence agencies do all of them.

Yes, very much this. Full circle back to the famous xkcd comic that's always posted in HN articles like this.

https://xkcd.com/538/

Perhaps we need to clarify our terminology. Typically when discussing HTTPS or TLS, we are discussing transport security, not end-to-end. I wouldn't classify either HTTPS or TLS as end to end, although I can see if you read my comment a certain way it almost seems as if I did.

That said, I am still unconvinced that governments have somehow compromised most end-to-end systems, especially not the ones that experts suggest work (PGP, Signal Protocol, OMEMO, etc). I should make particular mention that compromising and end-to-end system has nothing to do with whether or not a specific target can be made to give you their key(s). If you're being solely targeted by a billion dollar agency, regardless of their origins or motives, you probably won't be able to run or hide for very long. It's important to consider targeted attacks too, but let's not assume Hollywood scenarios here when discussing whether entire crypto-systems have been compromised.

Here's a recent Thinktank summary of the encryption and what access is difficult for the government.

https://csis-prod.s3.amazonaws.com/s3fs-public/publication/1...

You will find that the following summary is correct: The federal government does not have an issue getting the plaintexts it desires today.

This is because of a large number of capabilities come together: first most communications does not have unrecoverable encryption. Strong encryption is extremely rare, composing very small percentages of the communications available. Then, where encryption is available it isn't applied by default. Then, where encryption is available and applied by default, there are key escrow mechanisms for most of these cases. Then, there is a legal mechanism that requires communication service providers to provide decryption for any of the encryption that they provide. Then, what encryption can is build over the mechanisms for providers and services offer, most of it can be defeated. Then, what can not be defeated mathematically, access can be gained from other types of law enforcement activity, including lawful hacking exercises and capture of unlocked target devices. Then, what can't be gained by any of this can be inferred from metadata that is almost always available in plaintext, provided by mass surveillance collections.

I encourage you to read the associated literature, both reports like the above, and the contents of the Snowden Disclosures.

Namely: both law enforcement and intelligence operations are able to thwart the very vast majority of communications among Americans as well as those abroad. There do exist some systems that lead to unrecoverable encryption - they are in the very vast majority and can and are thwarted using other types of operations (if the cost-benefit analysis deems it necessary). And so it is not a reasonable position to remain skeptical in the face of overwhelming evidence.

It should also be noted that a number of "Hollywood scenarios" have been disclosed and have been seriously debated on a national level: before the 90s all cryptography in the United States was mandated to be weak to government interception (this isn't that long ago) and can we go without mentioning DRGB? Apparently we can't.

The sum total of all of this is that by and large, as a rule, federal intelligence and usually even regional law enforcement are able to access almost all civilian communications (in Seattle, for example, all of the cities communications including social media posts are hoovered up by our city police).

>This comment in particular seems to imply that government agencies can break e2e encryption.

I don't know about that, there is much speculation on all sides. However, I would say that if the FBI/NSA/CIA decides that you, specifically and in particular, are someone they want to surveil, there's probably not much the average person could do about it. Once you're finding yourself having to take photos of your desktops motherboard to make sure no one is replacing chips on it while you're at work, it's probably only a matter of time until you lose that fight.

The main point of wide-spread e2e encryption is to force such agencies to actually make those choices. The reason they're doing dragnet surveillance is basically because they can. We're handing them a buffet, no one should be surprised they're digging in.

The good news is that if we act en masse, we don't have to raise the standard of security by very much to make dragnet surveillance impractical. The HTTPS everywhere initiative is a great example of this. End users are having to do basically nothing, and yet we're cutting off a whole category of the dragnet in one go. If we can pull a similar coup in email, we'll have made real progress.

The legality of this is being litigated now.

https://www.eff.org/cases/jewel

You need to separate national security activity from traditional law enforcement activity.

Why? Parallel construction proves that law enforcement does not.

Does not what?

They don't separate national security from law enforcement. They _say_ they do, but they don't.

Parallel construction is when an intelligence agency tips off law enforcement about some nefarious actor (like say, Ross Ulbricht) and then LE has to construct a plausible scenario where their hard-boiled detectives tracked down the bad guy through traditional police work, not through spying on American citizens.

[1] https://www.washingtonpost.com/news/the-switch/wp/2013/08/05...

I suppose it's a good time to invest in hard drive manufacturers.

> They wave away violating the 4th amendment of the US constitution by saying that copying, storing, indexing, and searching through US citizens' private information is OK as long as a human doesn't read it.

This is funny because this is the same technicality companies like Google use to claim they aren't violating your privacy: That the only thing reading your email is their data mining algorithms, not humans.

That's a strained analogy. Its far easier to not use Google. I have to actively agree to Google's terms and conditions. OTOH I have to actively opt out (if at all that is possible).

Ideally in a democratic forum I would have some say in it, but that exists only as a fiction.

How many emails do you get from / send to gmail users? For me, it would be difficult to avoid using google because so many of my correspondents use it.

I am saying that legally disowning your country, extricating yourself out of NSA coverage etc would be more difficult.

> match a search query on a nefarious keyword.

Sorry this is not true. You are referring to an email chain involving FI maybe, and within that USPI would be minimized so that the analyst cannot see who the US one is. They would need to specifically override and would need a good reason to get USPI as that is taken seriously.

> "Full take" sounds incredible-- it entails recording and retaining EVERYTHING passing through the internet.

This is also not true. It sounded nice during the leaks and everyone loved to hear it, but this is simply not true, not plausible, and would be a huge waste of money because they would wind up with too much USPI (totally useless). The collection systems are mostly going to be getting traffic from MENA area, although the leaked information was vague regarding specifics, for obvious reasons.

(No argument with regards to ethics of bulk SIGINT collection, many opinions out there on that, just commenting as it is very unfortunate to see falsehoods repeated due to reporters poorly understanding the source material)