Dropship — successor to torrents?
forwardfeed.pl
forwardfeed.pl
So no, not a successor to torrents.
I wonder why the github repo has been taken down.
At which point does it become illegal? Is sharing it with one or two people ok? I would think that even putting it in your public folder is not necessarily illegal: What if you don't share the link publicly (or only with one or two people)?
Services like Rapidshare thrive on those ambiguities. They let you upload any file and give you a link, only after this link really becomes public will they take down copyrighted content (which introduces a time delay).
I have actually never seen that happen with Dropbox links (which, I think, is the right strategy for them: It would be bad for their brand if they were to become "that piracy website"), so they must be doing something different.
Actually, it could be. Copyright means exactly that: the right to copy.
As I said, copyright law is quite complex and full of exceptions and clarifications.
We're not Big Media people, but what about other content creators? Especially musical collaborators...
Myself, I really regarded dropship as a nice feature. As Dropbox had implemented the great idea of putting all humanity's data in one big hash-addressable vat, sharing is a logical extension. If you would cache the popular blocks locally (dropbox already does this in a way with LAN P2P), global data distribution would be pretty much a solved problem.
Obviously, this affects legal and illegal files in the same way. It's really a shame that people are still so obsessed with the illegal applications, that they become blinded to how useful this is for legal ones.
Even though there is a lot of (social) legal sharing going on between users, the focus is always on illegal sharing. He has a point there, though I think it's a pity.
IMO it's not even that suited to piracy, as the deduplication means that they can find everyone that has a file! Torrents are way better for that.
The principles of dropship could be used for sharing photos, videos, public datasets, git-like source control, or even as building block for wiki-like distributed databases. The possibilities are endless when every file can be called up with just its hash.
If they did how hard would it be to pad media files with some salt to break hashing anyways? Not hard at all...
Also, wouldn't breaking the hash nullify one of the ostensible advantages of this method (the de-dupe of the stored files)? If the goal is solving global file distribution, making each copy of every originally-identical file unique - and therefore requiring n times the storage - isn't a viable solution.
i guess storing something like a small truecrypt volume there would be just about enough.
"These utilities make use of the deduplication scheme of Dropbox__ to allow for "teleporting" files into your Dropbox account given only a list of hashes, provided of course that the files already exist on their servers. This enables arbitrary, anonymous transfers of files between Dropbox accounts."
Between this and the minor information leakage issue I suspect Dropbox will be making changes to their deduplication scheme.
A simple way to fix both of these issues is to require each user to upload the complete file once, regardless of whether Dropbox already has it stored. Deduplication in storage and per-user uploading is still possible.
Also interesting to note is the Github repo for this has been deleted. Tarball of the source is still available.
Additional measure at that point: the server could challenge the client to provide the values of bytes at a couple of arbitrarily chosen byte offsets of the original file. (Could precompute that, provided the queries don't repeat often).
Client A wants the file that Client B has so when Dropbox asks Client A for some random offset, Client A asks Client B in the background and relays the result to Dropbox.
It really depends on how far pirates would be willing to go.
For some reason, this inspired me to write a blog post: http://a3nm.net/blog/deduplication_attacks.html and http://news.ycombinator.com/item?id=2489594
For example if some web site charges per download of a file, but still has the hash posted publicly, you can try to "steal" it from someone who has it stored privately in Dropbox?
IOW, the file hash is equivalent to your account login/password combo [restricted to any given file]?
If you have a sub-4MB sensitive file, and you publish its SHA256, and the Dropbox protocol applies the hash function in the same way as file hashing tools (e.g. doesn't include a tag meaning "this hash is computed particularly for Dropbox deduplication" into the SHA computation), yes, then apparently people can download your file.
However, I rarely see SHA256 checksums along with download links; more SHA1 and MD5.
Another related attack could be to start with a known file (say, your employment contract), swap out the name with a colleague and generate a bunch of files with different salary amounts, essentially bruteforcing sha256 sums. If dropbox suddenly coughs up a file, you've revealed his salary!
Dropbox effectively acts as an "existence oracle". You can't ask it to cough up a file you don't have, but you can ask it if a given file exists anywhere in the system.
This would be an effective way for law enforcement or copyright civil enforcement to check for content that is clearly illegal or a certainly copyright violation to possess. They would need to query for a set of hashes of the given illegal content. If any matches returned positive data, they would be able to issue a subpoena for all users who stored the given content in their dropbox folder and pursue them further.
How can something be "clearly" a violation? If I have an album, but copy someone else's rip instead of making my own - is that "clearly" a violation? Alternatively if I used the same application, I'd probably obtain the exact same file - is that clearly a violation too?
(grooveshark kind of operates on the assumption that it's ok)
The following dropship file was assembled using only shasum, ls and vi:
{"blocks": ["f3f754a5dcd93f271ad013a5ee84f495a36da84f152e0a1fec4646345b0c10d6"], "name": "ostseestrand.jpg", "size": 514779}
Dropbox its deduplication scheme works by breaking files into blocks.
Each of these blocks is hashed with the SHA256__
algorithm and represented by the digest. Only blocks that are not yet
known are uploaded to the server when syncing.
By using the same API as the native client, Dropship pretends to sync a
file to the dropbox folder without actually having the contents. This bluff
succeeds because the only proof needed server-side is the hash of each 4MB block
of the file, which is known. The server then adds the file metadata to the folder,
which is, as usual, propagated to all clients. These will then start downloading
the file.
I wasn't impressed by the OP, but this is actually a really cool hack.
For example, Person A wants to distribute a copy of a CD or something. They upload the file to Dropbox normally. They then use Dropship to create something describing that file, which they then publish. Persons B and C download that descriptor and feed it to Dropship, which tricks Dropbox into thinking that they also own copies of the file. Dropbox then lets Person B and Person C download the file that Person A wanted to distribute, and mission accomplished.
It's all very clever. I like it.
Let's say you want to upload files A and B to Dropbox from your computer. A is a 3mb file, and B is a 12mb file.
The dropbox client first looks at A, sees that it's <4mb, and so hashes[1] the whole file. That means that it runs a function which turns the file into a 256-byte string (a "hash") which is unique[2] to that file.
The client then sends that hash to the server, which checks to see if it has already seen that hash. If it has, then it assumes that it already has the file, and just copies it from the previous location where it stored the block with that hash. If it hasn't, then it goes ahead and uploads the file.
The process for uploading file B is very similar, except that the client breaks it into three 4mb blocks, hashes each of those, and sends the hash to the server to see if it's already received those blocks.
Phew. OK, now we can get to why Dropship is (was?) a neat hack. The idea is, if Alfred has uploaded file C, and Barbara wants to get a hold of file C, but doesn't want to download it, she can just send the dropbox server the hashes for each 4mb block of file C.
The server will see each hash, say "ahha! I've already got the block represented by that hash, so I won't make you upload it!", and put the file in Barbara's Dropbox.
Does that make sense?
[1]: http://en.wikipedia.org/wiki/Hash_function
[2]: Not really unique, but the idea of hash functions is that we turn each input into a "hash" which is really really really likely to be unique, so likely that we can treat it as unique.
> This file is no longer available. For additional information contact Dropbox Support.
So, Dropbox has censorship? Ni-i-ice.
We have received a notification under the Digital Millennium Copyright Act ("DMCA") from Dropbox that the following material is claimed to be infringing.
/Public/laanwj-dropship-464e1c4.tar.gz
Accordingly, pursuant to Section 512(c)(1)(C) of DMCA, we have removed or disabled access to the material that is claimed to be infringing or to be the subject of infringing activity.
-----------------
This is BULLSHIT! Dropbox is censoring this because they don't want it to get out there. What will they censor next?
Wladimir did release the software under FOSS Expat ("MIT") license so he can't really take it back. It's now up to good will of others.
While I understand that this may put Dropbox in unfortunate situation, such methods to take down the problematic piece of software somehow feel wrong.
Like I said for file over 4MB it seems fine, guessing sequential hashes would be all but impossible. I assume the realistic solution is just to encrypt my files (preferably in a truecrypt volume over 4MB in size) if I'm truly concerned.
On a side note, it would be interesting to see if this could be modified to tell me how unique my overall file set is.
That has significant benefit over shared files and I have to think would scare the heck out of dropbox because of the ire it might bring upon them. This would have to be a worst nightmare for them. Although removing deduplication would solve it for them (with significant increase in what has to be stored).
1) User buys a file from the rights owner and explorers it into his Dropbox
2) User obtains a blockwise hash of the file and runs dropship on it
3) User obtains a public URL from somebody who has the file and downloads the file from the Dropbox web server
The point is that the Dropbox server cannot distinguish 1 from 2, but both from 3. Therefore, 2 should be more robust against takedown notices than 3.
In fact, I think this "feature" is one of the (many) reasons why Dropbox doesn't have an opensource client. And it isn't exposed it in its so-called "API".
Edit: I just saw that they killed the feature: http://news.ycombinator.com/item?id=2483053