Space Monkey: Taking the cloud out of the datacenter
kickstarter.com
kickstarter.com
How big is the hard drive in there? Judging from the price, the disk inside can't be much bigger than 2TB. That means if I have 1 TB of data and all the other have 1TB of their own data, it can only "replicated" exactly once (unless there are dedicated "cloud" servers involved), Which means, if my device breaks all devices which received a portion of my data need to be online for me to restore the content.
How does the encryption work? Surely the AES key can't be stored on the device - if it breaks your data is lost. If it's stored in the cloud it's not any safer than cloud storage.
1. All (or even most) users fully utilize their entire 1 TB quota. In reality, the vast majority of users will probably not even use half of their quota. By the time that changes, hard drive technology is likely to have advanced, and they can start selling larger boxes. Then the cycle repeats.
2. The data on the drives is not compressible. While people certainly could store only compressed files, that's not very convenient, and the reality is that most people will have at least some portion of content that is readily compressible.
Just by exploiting those two facts, the network can probably work without any external cloud assistance. But the smart way to design it would be to also have traditional cloud storage as a fallback. And that would be safer than traditional cloud storage, while still saving money since the fallback would only be used in special circumstances.
That's why sometimes if you upload a movie you downloaded, it uploads in a matter of seconds.
Though I suppose Dropbox could still use the same deduplication while making it harder to link a file to your account.
Edit: They changed how it works on the server side, without going into detail. It's discussed here: https://news.ycombinator.com/item?id=2483053
It will vastly increase reliability, though. I can think of many reasons why having it on a general purpose hard drive is a really bad idea (turned off accidentally, freezes due to bad apps, virus infection, etc, etc).
In general although I like the idea of an open source version of this I think the problem becomes so much harder as you don't have a central authority to prevent misuse of it and therefor have to build that into the protocol from the beginning.
Hmmm, I think this may be one of those wedges that gets stuck in my brain and ticks over for a while. Its quite a fun problem to try and solve.
If bitcoin can use a peer-to-peer system to come to a consensus on the validity of financial transactions, it should be possible for a peer-to-peer cloud storage system to come to a consensus on how reliable a node is, and how valuable it is to the network.
Uh... If it was on your computer's hard drive and your hard drive failed, it wouldn't be much use as a backup system, would it?
Interesting to see someone stepping up and delivering a consumer-friendly experience on top of such technology. It's incredibly useful but reliant on a healthy network of peers. A plug-in-and-forget solution with good clients that integrate well into the ecosystem of devices we use today, I think, is very clever.
I'd prefer to see something open-source but this is definitely a step in an interesting direction. Personally I limit my use of "cloud," services because I don't trust other people hosting things like my contacts list, calendar appointments, etc. I do see the benefit and convenience of such services though and if this device/service is capable of synchronizing more of that data from my applications between my devices I would buy into it in a heartbeat.
I hope the kickstarter goes well and this thing takes off (or something similar... again, open source is preferable).
It can be ameliorated by the cloud tricks of compression and git-like copy-detection across nodes (people downloaded the same movie, music, image, document, software etc) giving aggregate savings.
Of course, most people won't use their full quota initially, and meanwhile more adopters come online with fresh excess capacity... a kind of ponzi-scheme/chain-letter (to be fair, as drive prices fall over years, units will likely become 2TB, 4TB etc).
Good idea, but I question the scalability of the solution. How does deduplication work on remote devices if you have a lot of them. Is there a checksum of the file against a central server? Curious.
I could be entirely wrong, of course, someone correct me please.
Correct me if I am wriong but this is storing data as torrents and each device is a torrent client and store box?
I'm not sure how valid the torrent analogy is in this case.
Interestingly, the Tahoe people have their own product to offer:
... although I have no idea what state of production that is in ...
Not sure if I like this. $10/month for a TB is the same as Amazon glacier. Sure that's not the same since Glacier is for archival but still... It's not THAT cheap.
(b) If all you need is archiving of your stuff, and you don't want ongoing fast access to your movies, music, photos, etc., then sure. Glacier is probably a better fit.
The idea is to actually access these files regularly from more than one computer. Hence the web interface and the emphasis on mobile devices. It could also serve as a media library for various devices, not just in your home, but presumably anywhere.
Dropbox and plenty of others basically do this already, but $10/mo at Dropbox only gets you 100GB. And I don't think it does streaming, though I could be wrong.
Personally, I'm curious how they access the files externally. The device must act as a server, I suppose, but it seems like firewalls would present an issue to less technical people. Maybe they just assume those people would never buy this?
- options to pair up with specific other devices for the backups (family, trusted friends etc)
- a web app for posting status updates etc (implementing tent.io?)
Basically the things social networks offer now, but with you owning your own data.
Unless it's a way to draw out more money over time from the user?
But also yes, there are nontrivial ongoing costs to keeping the network working and working right. For example, we need dedicated online servers to allow NATted devices to talk to each other, coordinate storage use, and deploy security patches and bugfixes, and we want to add more features over time.
This seems like a great (nay, genius) idea if you just buy a device that accepts a standard 3.5 inch HD, half the space is yours, half is for distributed backup, no monthly fees, just the upfront $ for the box itself..
Upload, where bandwidth limits are the worst, is almost entirely based on replicating data you've added to Space Monkey.
(I have an alpha Space Monkey device.)
Deleted comment
How reliable is the network - will you be able to cope with Sandy style blackout?
Will there be a software component in which I designate a hard drive on my (always on) pc and skip the external device entirely?
How many times the data will be backuped?If only twice than if my home device get stolen/eaten by a rabbit/simply breaks then it is needed only one hardware failure somewhere else in the system to have stuff lost forever.
Regarding durability, the Kickstarter page mentions:
Q: Is my data safe?
A: Yes! Super safe. Here’s why: when you put files in
Space Monkey, you not only have a copy of everything on
your Space Monkey device, but each file is chopped up into
tiny pieces, encrypted, then stored to dozens of locations
outside of your home, in such a way that even if half of
those locations were destroyed, all your files would still
be safe.
_Normal Accidents_ by Charles Perrow
Please, please read this book RE: complex, tightly coupled systems.
They require hardware/software/network engineering, manufacturing, support, sales, and more scaling to support these products.
I still don't understand how exactly they store all the data since this is supposed to be removing datacenter overhead, not adding to it. From what i've read it sounds like the home devices themselves are the redundant storage for all the other customers. Which sounds terrible.
Bandwidth is a simple consideration. In America we really haven't caught up to the bandwidth of most other 1st-world nations. Some reports give the average download speed of American broadband users at 7.1Mbps. Netflix reports the highest average broadband speed is with Google Fiber, at 3.45Mbps. Large swaths of the country still operate on 1.5Mbit connections or less, with upload caps being more like 256Kb for those users. Due to the disparity in speeds we can assume most of our transfers from home users would be limited to relatively slow speeds, not to mention (supposedly) using idle bandwidth.
If you need your files, how long is it going to take to get them? Assuming something like 2.5Mbps down, and if there were five copies of your data transferred at 256Kbps, that's 160KBps. Downloading at that speed would take 77.6 days to download the entire archive.
Then there's redundancy to consider. Disk drives are limited in capacity. Currently 4TB drives are available, with 5TB by the end of the year. To give each customer 1TB of storage while still storing other customer data on the device, you have to decide first how redundant you want the data.
If you have a fancy algorithm you can store hundreds of clients' data on the device. Maybe they only have a couple small files, so they can be replicated in lots of places in a small amount of space. But eventually the files will grow in size, so you'll have to remove some redundant copies to make room for maximum capacity. Let's assume they use something like RAID-6, because RAID-5 is shitty for large capacity small arrays. Since we assume a 4TB hard drive, we can have N-2 storage, so for 4TB that's 2TB; make it N-3 storage and you can fit 1TB of user data and 3TB of random customer parity.
With maximum capacity use, if three replicated copies of your data go away, your data goes away. How likely is this to happen? This[1] article explains how as drives increase in capacity they aren't increasing in reliability, which results in faster unrecoverable errors. As time goes by it's more likely your data will be lost.
This is assuming your data is available. If their software engineers are smart they might develop an algorithm that can take into account actual used space and add or shrink the number of replications across the network based on availability. So even if you have used 1TB of data, as long as other people have extra free space, you can keep replicating copies of your data to more devices. Sounds good!
Let's say your data is replicated on five devices. One device holding your data fails and is sent in for repair. Another is taken offline because the owner's internet connection is taken offline by a storm. Yet another is available, but is on a very slow connection. Your data is now on two devices. Sounds fine.
In the event that your device is unavailable, you will be able to replicate your 1TB from the two people who hold parity copies of your data. How much data that is, and how long it takes to recover, and whether or not that transfer puts one of their devices over the limit for a URE, is subject several factors.
Of course, there's many more factors to consider here, and i'm not an expert in any of them. But before you trust your data to a $10-a-month service, ask yourself: How reliable is the drive, how fast is the connection, and how much do you really need your data?
[1] http://www.zdnet.com/blog/storage/why-raid-6-stops-working-i...
2. Restoring backups takes a while. It's just the way it is. But the client should be smart enough to get prioritized data first -- if you need to open a picture, it should fetch that first, so you can open it right away.
3. Files are broken apart and stored in chunks, so "growing size of files" is irrelevant.
4. The probability of your data being lost doesn't increase over time. The node is constantly checking the status of your backups. If a chunk is missing over a long period of time, it will back it up again. Redundancy is only part of the solution -- some missing chunks are recoverable through erasure encoding (like in the article you mentioned). Your node backs up to hundreds of other nodes, reducing risk of failure. Think of having 200 drives in an array instead of 7.
5. The article is somewhat FUD. Making projections for 2019 based on hardware speed where we're at now? Many embedded devices are already replacing ARM processors with dual core intel. As data sizes increase, our processing power will also increase.
2. Restoring backups should not take over two months. That's just not the way it should be.
3. I was talking about using up all of your allocated 1TB, not the size of individual files.
4. I suck at math and can't wrap my head around erasure codes. But think of hundreds of nodes in recovery mode: peak i/o load, bandwidth pegging, resulting in poor performance, higher heat, more likely URE failure. Ever use a cloud server where the neighbors are using too much i/o? That's what will happen to you when your device is pegged by serving file chunks to neighbors. Even with really good load-balancing you have to eventually choose between lower availability or swamping a node.
5. The article is based on UREs not increasing with the size of disks, leading naturally to more errors sooner. As capacity increases and URE stays stagnant, the problem gets worse. The projection will change if UREs get less likely, but there's no sign that will happen in the near future.
4. He was probably saying that, as the average age of the nodes in the network goes up, the chance of them failing goes up. Also, the duplication you say (hundreds of copies) is in direct opposition to usable space (hundreds of copies means you can only use 1/100 of the space on your node. Because of this, I don't think your files will ever be duplicated on the order of "hundreds", probably less than 10. Also, it's important to keep clear: My node will backup to hundreds of other nodes, but each individual file fragment will only be duplicated to 3 (or whatever) nodes. That smaller number is the important one that keeps getting discussed.
Space Monkey data will be resilient to considerably more than 3 nodes failing.
Other people's upload bandwidth only needs to come into play if your device isn't available: its hard drive breaks, it gets stolen, your house burns down, etc. In that case, yes, getting all of your data back probably will be slower than you want it to be, but you will get it all back.
Regarding redundancy, be assured that Space Monkey can provide much better numbers than RAID-6. We do assume that a decent percentage of devices will have poor network uptime, and we want your data to be available and redundant even in the face of that, even if the "fancy algorithms" take more processing power on the devices than we would need otherwise :).