How Dropbox sacrifices user privacy for cost savings
paranoia.dubfire.net
paranoia.dubfire.net
I could see why this would be more work to implement (you have to key on user+contenthash), but it would still be interesting to know how much Dropbox and its users actually benefit from cross-user de-duplication.
Per-user deduplication will mean, I need not upload the same file twice into my own account? What's the use of this?
I keep some of my 'paid for' software installables backed up in my Dropbox, and they tot up to ~1.5 GB (the Humble Indie Bundle games). When I started the upload however, it took maybe 5 seconds because of cross-user deduplication, and I am super grateful to them for this feature.
I imagine this feature saves users tons of bandwidth, as most of the people I know use Dropbox for backing up important software, rare music and videos.
It's not a line of argument. It's a line of inquiry. You've given anecdotal evidence that cross-user deduplication benefits you and people you know, but what about some actual numbers from Dropbox?
Producing actual numbers -- "eg cross-user deduplication saves our users 30% of their upload time and bandwidth, on average" -- seems like a great way for Dropbox to counter this issue.
> I imagine this feature saves users tons of bandwidth, as most of the people I know use Dropbox for backing up important software, rare music and videos.
We don't have to imagine! Let there be numbers!
Also -- "rare" music and videos that everyone's uploading duplicates of? ;)
Let's say right now Agent Bob or RIAA layer Cindy decides they want to know everyone in the country who has a copy of a file. There's no practical way for them to do that. But now, they can upload target.iso to dropbox, see that it uploads instantly, and all they have to do is get a court order to compel Dropbox to tell them all the other users who have that file.
Every user that has that file is now exposed. Dropbox is now a single point of failure for every user's privacy, and vulnerable to attacks from any legal court order -- and we've seen what can happen with the abuse of prosecutorial and judicial power. Copyright infringement's the obvious case, but when you start to consider how this ability to fish for files could be abused, and how tempting it'll be for people try to abuse it, it seems more serious than I think you're considering.
When I think Dropbox, I think sharing, I think convenience, I don't think backup and security. For backup I need more space, for security I need to use my own private key (not a password that one can change/recover). Neither of these things is offered by Dropbox. And this is the reason why I never confused Dropbox with, say, CrashPlan. One is a way to share and collaborate, the other is a place to send my private key encrypted bits to.
My individual privacy is not compromised by somebody being able to say if a certain file is stored by the entirety of Dropbox user base. The other claim, that, given a court order, Dropbox can be forced to turn over your files or tell the court if you store a certain file _may_ be true, but I don't think Dropbox, the company, has ever promised that level of security.
"What this means, is that from the comfort of their desks, law enforcement agencies or copyright trolls can upload contraband files to Dropbox, watch the amount of bandwidth consumed, and then obtain a court order if the amount of data transferred is smaller than the size of the file."
This, I think is significant, especially if Dropbox is advertising security, privacy and encryption. As the author mentions, the ToS are being updated to reflect the above possibility ("if Dropbox receives a warrant, it has the ability to remove its own encryption to provide data to law enforcement").
Except the key is actually kept under the doormat, because people keep going in and out and keeping it elsewhere is just too inconvenient.
If you told someone a room like that had military grade security, you would be called a liar. No matter how fancy the padlock. Without knowing the details of DropBox's setup I'm going to refrain from calling them liars, but this all seems pretty fishy to me.
My point is, DropBox advertise proudly on their website that they use military grade encryption to protect their users' data. However, it has now been independently shown now that the keys to this data are in DropBox's direct possession and are in routine, daily use, decrypting one person's data so another can access it (this is what happens when deduping allows you to download something you never actually uploaded yourself).
To me, this implies that their claims of "military grade security" may be unjustified and just yet another example of security theater in the cloud.
Without knowing the exact architecture of their system it's hard to say for sure, of course. But think about what the encryption they claim to use is probably supposed to accomplish. Then think about whether it actually does that if a large proportion of DropBox's servers and employees have access to the decryption keys.
[edit: Amazon store data on S3, so it is in fact important that they encrypt it (even with relatively relaxed key management) as they have no direct control over the infrastructure. I still don't think this meets the bar of "military grade security", but I guess that's marketing for you.]
If you want something to stay secret you MUST either:
a. Not put it on the internet/cloud.
b. Encrypt it yourself before uploading. Yes, there are trust issues with commodity encryption software as well, but these may be mitigated somewhat.
That's a pretty slippery slope to presume that a desire for privacy is tantamount to an admission of guilt. Why would you presume that simply because I have information I want to keep secret, I must be doing something wrong?
Conversely, is it ok if we install webcams throughout your house, since apparently you have nothing you'd like to keep private for non-criminal reasons?
I don't thing it is illegal though, is it? If I want to sync a bunch of music files that I purchased because I want to back them up, or access them from multiple computers, but I don't make those publicly available, is that illegal?
Genuinely curious, as I've been thinking about doing exactly this recently.
Why not compute the file hash on your local machine before encryption, and check that hash against a master dupe list (hash, dupe_count) of all hashes from all users' pre-encrypted local files?
Secondly, I cannot see how this requires there to be an index of users hashes. Surely one could store hashes with reference count, increment when a user adds, decrement when a user deletes. The user ID isn't necessary for a reference counter.
Not saying Dropbox isn't doing what he says. But he says de-duping proves they can decrypt and proves they have a list of who has the same files. I don't see it from de-dupe alone.
Or in order for you to download it through the web interface unencrypted.
Not necessarily. The client could send an encrypted version with only (plaintext) hashes of the pieces. EDIT: no, I'm wrong.
> Or in order for you to download it through the web interface unencrypted.
This one I will give you, unless they're doing something really weird like client side decryption through Javascript, which I'm not sure is even possible. However, they could in theory not store the key until you actually use the web interface (and you don't have to, so they wouldn't have it), and also not store the key when you do.
- A hash computed locally (on the clients with the large unencrypted file) and sent along to be used by dropbox to detect dupes.
- The key used to encrypt the large file is some function of the file, but not of the hash. The important point is that it's not encrypted with a client specific key, but rather a file specific one. Thus if you have the file, you can compute it.
- When a dupe is detected, the server requests that the uploading client send it a copy of the key, encrypted PGP so that only the other intended clients can decrypt it
I think that should work.
What do you reckon this is? IIRC, based on the other Dropbox article this week, the only relevant information stored in local Dropbox preferences is your account ID and a unique API token for the specific machine.
There was no mention of a plain-text password being stored locally.
That's actually an extremely interesting idea. I wonder if using some function of the data, f, to determine encryption keys, leaks information about the encrypted data? My armchair guess would be yes. Of course, the leaked amount (well, the slight-non random distribution by the key being a function of the data) might be small for large files/good function f, which would mean it's probably okay?
I've never heard of any research of a cryptosystem that works like that though, so I'd appreciate if anyone could provide some expert input (or maybe we should just email bruce schneier)
More generally, cross-account deduplication always makes it possible to tell if anyone has uploaded a file before - that's the entire point!
AES might not be suitable for this purpose, but I don't see why the scheme I proposed is necessarily weak. Do you? Anyway, I agree it's not particularly practical - if you're really concerned about data security, then encrypt before you upload.
The server cannot read files, but can determine who has uploaded any one cipher- or plaintext. ("Encrypted or unencrypted file".) I think that's still bad.
You could do this, but it would still be possible to determine which users have a copy of a particular file (or a piece of a file).
> Secondly, I cannot see how this requires there to be an index of users hashes. Surely one could store hashes with reference count, increment when a user adds, decrement when a user deletes. The user ID isn't necessary for a reference counter.
On the surface, it looks like this would discredit the first claim that I've just made. I think though that in reality it could be detected. For example, the Government could require them to wait and watch until a user downloads a file (or piece of a file) keyed by the hash of the piece whose owners need to be identified. Given that this is feasible, I don't think that there is any point implementing this measure, and it would help to maintain data integrity by not doing it.
This means it's theoretically possible for parts of the file to come from different sources, which means contraband files are 'built' from parts of otherwise legal files.
All other syncing services do things a pretty similar way
> public links for files easily done by creating an unencrypted copy
> shared folders you can do this by copying, encrypting with new keys for each shared user, and then sending notification of those keys to the user (via some side channel), and then deleting them on your end after they are accessed (you can re-encrypt with the accessor's keys at this point). You can do a bit better if you leave "half open" asymmetric channels (so you can store encrypted messages/keys only the recipient can decrypt), but that might be overkill.
> web access you probably will need to use java (or NaCl) to do this, as javascript tends to be too slow to do asymmetric encryption (the the needed bignum support just isn't there). If you're willing to wait, it can be done in javascript in ~5 seconds on a desktop PC.
> mobile access Uh, all modern smart phones can do encryption fast enough (well under a second).
> password recovery features This is the only salient point. You can kind-of counter it by using the security questions to encrypt the actual encryption key a second time, so that the data can be decrypted if you answer the security questions. But that's obviously less secure.
E.g. right now I assume a dropbox user owns a list of file ids with some metadata (e.g. that user's name for those files). If follows that if the government decides file XYZ is illegal then anyone with XYZ in their list is in trouble.
The user account could keep track of the total size of all the user's files and use arithmetic to keep it up-to-date, but not actually store the size of individual files except when they are "looked at".
So then the user's password (say) which is not itself stored is used to unlock stuff in the user's file table on a per request basis -- i.e. the actual file ids are only computed as needed. The actual mechanism doesn't need to be terribly secure, it just needs to be deniable. In other words without the user's password we simply cannot unambiguously determine which files are his or hers.
Even though it isn’t spelled out, I’ve always suspected that many actual backup services such as Backblaze don’t know the key if I decide to encrypt my backups.
You're right that sharing folders between different accounts would require having a key shared between the clients and thus stored on the Dropbox servers as well.
However, they could generate the key for the shared folder, give it to you and your buddies and yet store it encrypted using your master key generated from your username and password. Then it would be accessible to you, but Drobox would not be able to decrypt it without getting your password.
It looks like they're not doing that, but hypothetically, they could. And the examples you cite would not be impossible to deal with.
Of course, this would require much more work, would be tricky to get right and they'd have Thomas Ptacek on their back for using JavaScript crypto in the browser.
This just seems like the type of thing that someone much smarter than I can and will exploit in the future.
But it seems to me you would need to know the exact contents of the file in question to get that to happen, making the point moot. Perhaps I'm wrong on that.
If implemented correctly, the additional constraint on filesize being the same is irrelevant. Given one particular hash value, the probability that a second file hashes to the same value is 1/(range of hash function) if the hash function is modeled as an ideal hash function.
Relavent to the discussion: http://stackoverflow.com/questions/622930/purposely-create-t...
http://www.mscs.dal.ca/~selinger/md5collision/
'Extension attacks' are something else, which let you turn one collision into more, or create valid hashes for combinations of unknown text plus a chosen extension – not find an initial collision. See:
http://en.wikipedia.org/wiki/Merkle%E2%80%93Damg%C3%A5rd_con...
The 'length extension' property can be helpful, once you find a collision based on 'random' nonsense, in extending that into two documents that are each meaningful-but-different and still colliding, as was done in this 2005 MD5 collision demonstration:
http://replay.waybackmachine.org/20050612011328/http://www.c...
Of course every extra bit that has to be matched makes collisions 'harder' but length bits are much weaker than other options, except insofar as they may already be available for other reasons.
Popularity is also proportional to chance of being targeted by hackers and approached by government or corporation representing intellectual property owners.
At least Dropbox gets the endpoint-to-server encryption right.
Arguably the best feature of Dropbox for me is binary diffs. If you encrypt the Dropbox this goes out the window, or at least becomes significantly harder to pull off. Am I wrong?
How about comparing a hash of the encrypted data?
But wait, that never happened because it was a dup.
That means they can, and will, decrypt somebody else's data to serve your request. Cute, huh?
It will be quite valuable for users of services like Dropbox, Amazon Cloud Player, etc.
And sneakier still, a remote hosted bit torrent client that automates that process for you in a way that can provably show you never downloaded the copyrighted file, you just had a number on your drive, which if interpreted as a SHA256 hash key happened to identify a duplicate of the copyright file on Dropbox... (I'd be spectacularly impressed if someone managed to make _that_ trick fly on court in a precedent setting manner!)
I hope it doesn't happen, because that'd be a bit of a nightmare for Dropbox. They could blacklist known pirate rips of movies and the like, but what do you do when you receive a takedown notice for DRM-free content that some users are legitimately storing, but Dropbox is now (accidentally) illegally distributing? Could be the end of global de-duping.