Hacker News Reader

Comment by bigbrooklyn | Hacker News Reader

If you NEED encryption, don't use email.

From: https://blog.fastmail.com/2016/12/10/why-we-dont-offer-pgp/

What's the tradeoff?

If the server doesn't have access to the content of emails, then it reverts to a featureless blob store:

    Search isn't possible
    Previews can't be calculated
    If you lose your private key, we can't recover your email
    Spam checking on content isn't possible
    To access mail on multiple devices, the private key needs to be shared securely between them

update: want->NEED

View on HN

> If you want encryption, don't use email.

That's total nonsense.

> Search isn't possible

It absolutely is, in both theory and practice. The server stores an encrypted index, and the client walks it (requesting parts as needed). It's going to little slower, and a lot more complex but it's doable.

> If you lose your private key, we can't recover your email

This is a damn feature. I had my icloud account social engineered (someone walked into an apple store claiming to be me and they couldn't get their iphone syncing to "their account"). I'll never again trust another company with my private stuff.

> Spam checking on content isn't possible

This is probably your best point. It's definitely harder to do well

> To access mail on multiple devices, the private key needs to be shared securely between them

This is a non-issue. It can easily be derived from a password

Also search that only uses email titles is still at least 70% as useful as full title+body search. So I wouldn't count this as such a terrible non-feature of e2e encrypted email.

Only if you leave juicy content in the unencrytped subject.

Also, "What? subjects and recipients are unencrypted!?" -- Every User Ever

You can encrypt the metadata on the server, and enable local search on the metadata to avoid keeping the full mailbox on a phone, for example

> > Spam checking on content isn't possible

> This is probably your best point. It's definitely harder to do well

I think it's possible, just slower and more complex (like search) - and would have to occur upon unlocking your inbox.

You can mostly get rid of spam by requiring the sender to perform a proof of work if they aren't in your contacts list. I.e. whitelist of senders + proof of work or some kind of configurable per domain quota/proof of work.

I guess that gets rid of all other email as well..

How often do you get email from somebody that you've never gotten email from before?

Exactly once for each contact that has ever sent me an email

Maybe a two-way 'add contact' feature would be useful, like a facebook friend request.

Sure but then it's not standard e-mail any longer.

I remember having a server side system for generating cryptographic email aliases along the lines of:

base64encode(hmac("new-sender@example.com+valid-to+01012018", key))+user@mydomain.com

Truncated to something like fvv544+user@mydomain.com that would only be valid when sent with the from-adress new-sender@example.com (along with some clever magic to avoid email loops! :-)

I think maybe it was authored by ESR (Eric Raymond) in python - but Google only turns up various dkim schemes...

New senders would have their mail held back, and get a "please reply if you are human"-message - a reply (to the "magic" reply-to alias) would release the held mail and whitelist the sender.

A greylist variant of sorts.

I'm talking about now going forward. There are plenty of ways to make a person opt-in the first time they send you something. That process gets rid of more than 99% of spam.

This feels... parochial.

It wouldn't be so good if email turned into Facebook, where you can only contact somebody if the circuit has been established beforehand (i.e., both parties friend each other). What happens if this is the point of initial contact, and there's no other way to get in touch? I have to conspicuously friend this person from my otherwise dormant account and then not think about whether they're going to feel weird when I unfriend them later on?

I sent an unsolicited email a couple weeks ago. Academic-type, self published, personal webpages where I saw something that needed fixing, and so I did. There was no CMS, just a static site—and not the sort where you're running markdown sources through a generator and have a whole Git hosting service apparatus intertwined with hooks sunk into it. Plain, legacy HTML. I saved a local copy to my machine, made my changes, and mailed them in. They happened to be machine-readable patches, given the recipient, but it could have just as well been a message written in natural language. The changes were made, I was thanked, and done.

I'm still upset that in the 90s I could get in contact with almost anybody by looking in the phonebook, but when I tried doing that a few years ago for an old coworker, it was hopeless. Getting the listings for a city you've moved away from can prove to be harder than it seems, even when you know they have a landline.

These are the kinds of things you lose when you assume the world is already fixed in the form that it should maintain going forward.

From every client that a former client has sent my way. 50% of my clients introduce themselves with some variation of "Something happened, Bill gave me your name. Help!".

Does anyone actually use e.g. hashcash though? I love the concept, but it's useless if nobody can use it.

I don't think Spam checking should be a concern since most spammers aren't going to encrypt their spam.

Might even flag not just plaintext email, but quarantine all mail from unseen senders / public keys. Should make spam filtering much easier.

How often does spam come encrypted with your public key anyways?

Much more often if this is used as a spam measure.

Maybe. Still requires a re-encryption of the symmetric session key for every recipient - effectively proof of work.

"stretching" the equivalent of rsa(session_key, public_key) might be feasible?

Fwiw search that way is (I think) patented by Proofpoint. So it might be hard to implement.

It is smart to not speculate on patents. You have now possibly poisoned everyone reading this thread.

If patent law worked like coodies that comment would be awesome.

The interpretation of IP law, at least internally at many big software companies, does in fact work like coodies. Hence things like "clean room implementations" of algorithms, modules, API interfaces etc.

Clean room matters for copyright. It doesn't help for patents, as patents protects the idea, not the expression of it so expressing the same idea in a different way doesn't help. With a patent you want to know the specifics of the original to ensure you make yours sufficiently different to sidestep the claims.

Where not knowing helps with patents it is in that if you infringe, wilfull infringement increases damages up to threefold.

Willful infringement allows the court to give judgements for up to triple compensatory damages in the US, so sometimes not knowing can be valuable.

These are just minor problems. End to end encryption should not be forced upon users, but should be used selectively. You don't want to lose your entire inbox because you wanted to send a few encrypted emails.

> The server stores an encrypted index, and the client walks it (requesting parts as needed). It's going to little slower, and a lot more complex but it's doable.

How slow would this be with thousands of emails on a mobile device ?

> This is a damn feature. I had my icloud account social engineered (someone walked into an apple store claiming to be me and they couldn't get their iphone syncing to "their account"). I'll never again trust another company with my private stuff.

Most email providers don't have retail stores where this type of attack can happen.

> This is a non-issue. It can easily be derived from a password

What happens if this password got into the wrong hands or even lost? Is it possible to change? Must I re-encrypt every single email ?

> Most email providers don't have retail stores

Lots do, though, and the ones that don't often offer customer support over the phone. And anyway, social engineering doesn't only happen in stores or when talking to CS.

Really? Name any of the top 10 email providers that have stores, or a responsive 800 number for email support. I'll give you Apple, but they are the clear outlier. MS, Google, have stores, but not with email support, and questionable phone support at best for any non enterprise product.

AT&T? I don't have a list of top 10 email providers, but them and Verizon and Yahoo have all had people report social engineering attacks against them to gain access to accounts.

> How slow would this be with thousands of emails on a mobile device ?

Not that much slower than email is now. A B+tree index of (for example) 1 gb of total content and index is only about 1-2 gigs in size. You could just store that shit on your phone encrypted. Or 1-5 megs of index on your phone, with 2 round-trips for any email/first page search result per word.

> What happens if this password got into the wrong hands or even lost? Is it possible to change? Must I re-encrypt every single email ?

Overall yes, re-encryption is necessary. But that can be a batch process done while your phone is charging and on wifi over night. Is a drawback, but re-keying always is. But just because you got to change the keys every once in a while, doesn't mean you toss the locks.

> Overall yes, re-encryption is necessary. But that can be a batch process done while your phone is charging and on wifi over night. Is a drawback, but re-keying always is. But just because you got to change the keys every once in a while, doesn't mean you toss the locks.

Encrypted hard disks utilize a simple scheme whereby the password/key derived from the password decrypts a small set of keys or single key that's just randomly generated, and decrypts the rest of the drive. So, to change the password, all you need to reencrypt are the real keys.

  Password -- decrypts --> master key -- decrypts --> rest of hard drive
                           ^^^^^^^^^^
                           this is what gets re-encrypted on password change
                           and is small; ~a few KiB

Similar could be had for email.

The approach above also lends itself well to supporting multiple passwords. (If you share an account w/ someone, for example, though email has other tools like mailing lists that might be better suited.)

> How slow would this be with thousands of emails on a mobile device ?

This in itself is not a problem. Btrees are O(log n) for search, so thousands of emails can be looked up in 3-4 requests with the most naive way to implement remote tree walk.

The fact you're revealing the structure and index contents to the server as you search would be a bigger issue.

Regarding Apple and security, their policy via AppleCare seems to be to ASK (over the phone, for instance) for your cleartext computer administrator password before you send in your laptop for repair, without any warning whatsoever of the implications.

I spilled soda on my mac once, and took it to the repair shop. the receptionist there asked me for my password. I laughed and I said of course not. she was shocked and asked: well, how are we going to test the new keyboard. I don't know maybe try to type random things in the password field?

What if they install the keyboard but the drivers fail? Won't they need the admin password in order to complete the job?

If a keyboard needs drivers other than USB-HID, someone's doing something wrong.

I thought that too, until i bought a cheap netbook that came with windows (7? Student edition? I cannot remember). I plugged in a mouse and windows said it needed to connect to the internet to download the driver for my MS optical mouse. I practically fell out of my chair laughing.

The "driver" is for adding a gui for doing things like customizing buttons and sensitivity and stuff, not to make the basic mouse work.

Except the mouse wouldnt work. Either the os couldnt use it, or wasnt letting me. Either way, it was funny seeing one ms product not recognize another.

Many laptops use an SPI touchpad device, so they don't need to go through the relatively expensive and complex USB stack.

They have physical access to the device. They can enter recovery mode by holding down command r when restarting. It gives them a different copy of the os with multiple apps that you can type in (like the terminal) and would allow superuser access to whatever they wanted, however your encrypted home directory would remain encrypted and they would not be able to read it. In the olden days you would stick in a recovery cd and reboot onto that... you could also (historically and presently) stick a USB drive into the thing and boot into an os on that.... NEVER give out your password.

Or just boot into single user mode.

I had this EXACT experience just this month. I went to have a screen replaced and I even explained (and apologized for the inconvenience) that I was very security focused. I set it up in advance so that would perform the repair in front of me, that the device wouldn't be plugged into any of their computers, and it wasn't to be taken out of sight.

Then, he straight up asked me for my password, "most customers write their password down so we can test that it works."

I feel a little bad because the look I must have given him was pretty absurd. I told him no, I'll just take the risk and test it myself.

> > Search isn't possible

> It absolutely is, in both theory and practice. The server stores an encrypted index, and the client walks it (requesting parts as needed). It's going to little slower, and a lot more complex but it's doable.

Are you suggesting that to search your mailbox, the client should download every single encrypted message in the entire mailbox and decrypt them all locally to search them?

If not, how does the server get this "encrypted index" without having your private key?

> > Encrypted Index

This is not the same as the content.

Well, the client has to fetch message once, decrypt it, and upload updates to the index (kept on server).

I guess, it is a reasonable sacrifice for privacy, unless you want searches to be able to search within attachments including documents in big archives.

It's still likely too large to really consider for mobile use, but yes - far better than downloading everything.

The search doesn't happen on the server. The client (e.g. desktop client) indexes the messages and encrypts the index. Then, when another client (e.g. mobile client) wants to search for a message, it downloads the index, searches it, and then pulls down the appropriate message(s).

If an attacker can tell which part of the index was modified, that gives them enough information to decrypt the index and e-mails.

Clients would always have to download+upload the full index (which needs to be re-encrypted with a new IV). This is a huge problem - the index can easily be hundreds of MB for a large mailbox.

Technically if a client has a full index version X (in plaintext), it could modify X to get X+1, compute a binary diff between X and (X+1) - encrypt and upload the diff.

Another client on index version X could download the diff, and get index (X+1).

Some desktop client should probably do compaction from time to time.

I'd you are using a new IV when encrypting the new index then this won't work, since the old and new indexes will be completely different.

You would have to re-download the index after compaction. But the index and patches would be independent - you apply the decrypted patches to the decrypted index.

I'm not quite sure what you're getting at. It sounds like you're describing a known-plaintext attack, which modern ciphers are not vulnerable to. And what you are describing makes it seem like full disk encryption would be totally useless, but we know that's not the case.

So what, you hash each word in the e-mail and search for the hash, and this returns which emails include those hashed words? Would that be horribly insecure? I guess it would be impossible to salt those hashes, and it probably risks defeating the whole crypto.

> Would that be horribly insecure?

Yes.

> I guess it would be impossible to salt those hashes, and it probably risks defeating the whole crypto.

Exactly. (There are other problems too, but that one by itself is a show-stopper.)

https://en.wikipedia.org/wiki/Salt_(cryptography)

You wouldn't use a hashing algorithm to build the index. They are talking about an inverted index in a binary format, something like what lucene outputs. That binary index would be encrypted with a block cipher (AES, Blowfish) using a secret key and would then be stored on the server.

The mobile client comes along and downloads & decrypts the index in memory, searches it for some terms(s), the index returns 10 result, of which, the user selects one and the mobile client downloads and decrypts to show to the user.

Adding to the iCloud story, I forgot my security questions and I was able to have the AppleCare agent over the phone reset it for me after talking about what apps I've purchased recently.

I have experience at Apple with Account Security and that's a clear violation of SOP. That's a coaching opportunity for the advisor.

> I have experience at Apple with Account Security and that's a clear violation of SOP

It should be technically impossible. If it's a matter of choice for tech support reps, it's not secure for many reasons.

It's icloud, not personal device storage. It has to be possible if you want recoverable content. If you don't like it, you can back up locally to itunes.

> The server stores an encrypted index, and the client walks it (requesting parts as needed). It's going to little slower, and a lot more complex but it's doable.

Going on a tangent, but do you know of any services that offer such a thing?

https://cryptag.org

That doesn't seem to have any email capabilities...

> > To access mail on multiple devices, the private key needs to be shared securely between them

> This is a non-issue. It can easily be derived from a password

How does that change the equation? You're still exposing the thing-that-decrypts to multiple devices, thus (many!) more threat vectors. Lose one, and you lose them all, which is the point of the claim.

...so use unencrypted email?

The point is, some is better than none. More is better than some.

So many people here pointing out holes that make it worse than a theoretically perfect system even though it's leagues ahead of where we are now.

No, the (implicit) point in the start of the thread is that multiple other encrypted mediums don't leak as much. Use them instead.

If you care enough to use encrypted email, you should probably seriously consider abandoning email. (signing is different - that's proof of identity and non-modification, useful in many non-private scenarios)

>I had my icloud account social engineered (someone walked into an apple store claiming to be me and they couldn't get their iphone syncing to "their account")

Were you using 2-factor?

Search is still possible. You can stem words and store their weights without storing the actual unencrypted text. This isn't perfect security, but it's good enough that it would be difficult for the government to successfully use the search metadata in a case against you without a lot of other evidence.

This is what we do on FWD:Everyone for email threads shared within private repositories.

I've never tried it but I'd bet you could use instead use a key derived from the master somehow to salt a hash for all the stems too. I think that'd let you still do the search and possibly have a good index for it still but it'd add some hurdles to the whole thing so that you couldn't make a lot of good guesses about the actual contents of the emails. It would still leak information a out related emails (say there's a thread discussing a unique term in it), but it'd still be better than otherwise.

In theory you could HMAC all the stems, but at that point you really need to ask yourself what threat model you're trying to protect against. E.g. keeping only the stems is more than enough to prevent competitors from learning anything useful about your business if your database gets compromised, but using an HMAC probably isn't going to significantly slow down a state-level actor who has a warrant or zero day for your web servers. So on the balance I think the technical debt that would be introduced would likely make the overall system less secure, at least for most use cases.

DO NOT DO THIS. Statistics is more powerful than you'd expect. Also, the wrong rare word could absolutely be grounds for a very intrusive warrant.

Security is rarely good or bad in an absolute sense, but rather is judged by considering the assets under protection and the threat models you're protecting against.

Just because certain people need the sorts of protections provided by Lavabit doesn't mean that products like Gmail are bad or insecure. They're each secure for the use cases they're targeting.

I would argue the biggest downside to encrypted email is the sender, recipient, date, and size are known to all intermediaries.

First of all, sometimes knowing who you talk to is more important than knowing what you say. Traffic analysis lets you draw all kinds of conclusions.

Secondly, there are two parties, so double the chances for rubber hose cryptography. An opponent can approach either party and ask for details, so both parties are trusting the other can't be broken by the opponent.

Of course, we should all be encrypting everything anyway, no point in giving out free message bodies.

If you don't trust the other party you should not be sending him sensitive data. He has to be able to see it so of course he will be able to reveal it.

It is not that you don't trust your party, but that anyone with access to the servers or network between you can learn about who you are communicating with and when. Such metadata are often more useful to use against someone than the actual content.

Well, you could always use an old school local email client and would get search and previews for free. And some 3rd party not being able to "recover" my private correspondence sounds more like a feature than an issue.

Gee, if only we could execute code on the client! That would be very empowering in this situation.

I guess it's not possible, then.

This is all ridiculous. Download, index, and process your mail locally. Problem solved.

If we had more systems capable of resisting a local search and seizure this might be the case but as it stands almost none do.

Oh no, you're describing the solution rather than the problem: a global search, copy, and seizure converted to a local one that happens on per-target basis. Way better than mass collection with FISA warrants and systems like QUANTUM.

Is QUANTUM the Swedish one or is there a new one?

It's the NSA system that operates globally that can automatically fire attacks at systems based on patterns the operations people put in. Almost all the European countries are part of a SIGINT-sharing alliance per one slide. The exceptions were Switzerland, Iceland, and one I can't remember.

Search and previews should be possible on the client side, but then you need a standalone app, not a web interface.

possible, but very expensive even on small datasets

I'm downloading my mail from POP3 servers, so I'm searching locally. I'm copying access password to my tablet and phone (K9 client) and I don't keep more than a few dozen of messages there because the POP3 mailboxes are cleaned up when I download from the laptop. I backup my mail to a remote server, encrypted with duplicity.

I understand that this is not acceptable for 99.99+% of people, even the technical ones, but I think I could use a fully encrypted mail store. No problem with the mail provider ending up as a blob store: privacy-wise it's what they should be anyway. No harm to their business, if all the money they make are from users and they're not selling data.

The only problem is centralized spam checking. Running an antispam engine locally wasn't very effective years ago and Thunderbird was sub par. Is there anything that's on par with email providers right now?

Don't forget "authorities can't snoop your mails and will take us to court because we can't decrypt it."

> Spam checking on content isn't possible

But it's still possible to fight spam based on other features: https://atscaleconference.com/videos/how-whatsapp-reduced-sp...

>Spam checking on content isn't possible

How does encrypted spam work? Spammers encrypt message with your public key?

At the moment, encrypted spam doesn't exist. But only because there aren't enough people using encryption to make it worthwhile for the spammers to invest time doing it.

Theoretically, a spammer could scan the public keyservers for email addresses and public PGP keys and then send encrypted spam to all of those people. If ever a well targeted spam mailshot was sent, that would be it. But would you really want to get on the bad side of thousands of tech nerds?

I think email needs to stop being treated like a database. It should be used for sending and receiving messages, the rest should be handled by other software... like a database for example.

I personally think it's funny that people mix secure communication and email. Secure system should be built to be secure. Email isn't a good option for that. Most systems are designed to be insecure and email is just a great example.

Reply on news.ycombinator.com