Dropbox’s Exodus from the Amazon Cloud
wired.com
wired.com
There is a necessary abridgement that happens in media like this, wherein a few individuals in lead roles act as a vignette for the entire effort. In particular, on the software team, James and I are highlighted here, and it would be easy to assume we did this whole thing ourselves. Especially given very generous phrasing at times like "Turner rebuilt Magic Pocket in an entirely different programming language".
The full picture is James and I were very fortunate to work with a team of more than a dozen amazing engineers on this project, that lead designs and implementations of key parts of it, and that stayed just as late at the office as we did during crunch time. In particular, the contributions of the SREs didn't make it into the article. And managing more than half a million disks without an incredible SRE team is basically impossible.
My only disappointment is that by trying to set a PR friendly narrative, Dropbox greatly compressed the time spent on the project from inception to completion, and that erases a lot of people's involvement. As someone who worked on MP when it actually was a four person project, it's a little frustrating to finally be able to talk about it after all this time while at the same time the public perception is that the project began after I left the company.
One of the things that's not immediately apparent in an initiative like this is the huge amount of effort required to provision, deploy and manage a system of this scale, across a whole bunch of different teams. We couldn't have done any of this without a tremendous group of SREs with the wisdom to invest heavily in automation, in addition to all the hard work.
We'll follow up in a week or two with a blog post highlighting some of the work here. I think the framework for disk remediation (automatically detecting failures, re-provisioning, etc) is particularly interesting.
https://www.google.com/search?q=devops+job+title&ie=utf-8&oe...
My current role was sold to me as a "devops" role with the usual jargon of automation - I can't think of how this role has actually prepared me to pass a serious interview as an SRE at Google, Dropbox, Yahoo, or other tech companies. What really has happened after looking at all the recruiter spam and some of the interviews I've taken on a lark is that the term has become another way of marketing to engineers with modernized skillsets when they can't retain good ones for more than maybe two months (during which time the engineer is interviewing for another role almost certainly) because good engineers tend to run away from these companies for many legitimately founded reasons.
This number is very interesting. Basically Diskotech stores 1PB in 18" × 6" × 42" = 4,536 cubic inch volume, which is 10% bigger than standard 7U (17" × 12.2" × 19.8" = 4,107 cubic inch).
124 days ago Dropbox Storage Engineer jamwt posted here (https://news.ycombinator.com/item?id=10541052) stating that Dropbox is "packing up to an order of magnitude more storage into 4U" compared to Backblaze Storage Pod 5.0, which is 180TB in 4U (assuming it's the deeper 4U at 19" × 7" × 26.4" = 3,511 cubic inch). Many doubted what jamwt claimed is physically possible, but doing the math reveals that Dropbox is basically packing 793TB in 4U if we naively scale linearly (of coz it's not that simple in practice). Not exactly an order of magnitude more but still.
Put it another way, Diskotech is about 30% bigger in volume than Storage Pod 5.0 but with 470% more storage capacity.
That was indeed some amazing engineering.
2.5" 15TB ... and SSD, and expected to get a lot bigger too!
Also not everyone wants to be packing a petabyte into a box. At that level of density you need to invest a lot of effort in replication strategies, tooling, network fabric etc to handle failures with high levels of availability/durability.
What is your price per GB raw?
We have flash caches (NVMe) in the machines, but the long-term storage devices are spindle--PMR and SMR.
> What is your price per GB raw?
I cannot disclose, exactly, but it is well below any other public price I've seen.
Obviously if you are playing seriously with VMs and databases and whatever else both of those (SLOG/L2ARC) may become important for you, I'm going for the "i'm just using this to store my raw-format photos, backups for the taxes and other big files" kind of usage here. :)
That is, the better your software can handle repair, the smaller your failure domain can be?
A larger failure domain would only make sense if you wanted to minimize the compute required per unit stored?
Then again, I don't really like putting too much weight that high up in a rack that already has 2600lbs of gear in it.
The dimensions given are just an approximation.
Could you comment more generally on what advantages Rust offered and where your team would like to see improvement? Are there portions of the Dropbox codebase where you'd love to use Rust but can't until <feature> RFCs/lands/hits stable? Are there portions where the decision to use Rust caused complications or problems?
> The article makes a brief mention of Go causing issues with RAM usage. Was this due to large heap usage, or was it a problem of GC pressure/throughput/latency? If the former, what were some of the core problems that could not be further optimized in Go?
The reasons for using rust were many, but memory was one of them.
Primarily, for this particular project, the heap size is the issue. One of the games in this project is optimizing how little memory and compute you can use to manage 1GB (or 1PB) of data. We utilize lots of tricks like perfect hash tables, extensive bit-packing, etc. Lots of odd, custom, inline and cache-friendly data structures. We also keeps lots of things on the stack when we can to take pressure off the VM system. We do some lockfree object pooling stuff for big byte vectors, which are common allocations in a block storage system.
It's much easier to do these particular kinds of optimizations using C++ or Rust.
In addition to basic memory reasons, saving a bit of CPU was a useful secondary goal, and that goal has been achieved. The project also has a fair amount of FFI work with various C libraries, and a kernel component. Rust makes it very easy and zero-cost to work closely with those libraries/environments.
For this project, pause times were not an issue. This isn't a particularly latency-sensitive service. We do have some other services where latency does matter, though, and we're considering Rust for those in the future.
> Could you comment more generally on what advantages Rust offered and where your team would like to see improvement?
The advantages of Rust are many. Really powerful abstractions, no null, no segfaults, no leaks, yet C-like performance and control over memory and you can use that whole C/C++ bag of optimization tricks.
On the improvements side, we're in close contact with the Rust core team--they visit the office regularly and keep tabs on what we're doing. So no, we don't have a ton of things we need. They've been really great about helping us out when those things have sprung up.
Our big ask right now is the same as everyone else's--improve compile times!
> Are there portions where the decision to use Rust caused complications or problems?
Well, Dropbox is mostly a golang shop on the backend, so Rust is a pretty different animal than everyone was used to. We also have a huge number of good libraries in golang that our small team had to create minimal equivalents for in Rust. So, the biggest challenge in using Rust at Dropbox has been that we were the first project! So we had a lot to do just to get started...
The other complication is that there is a ton of good stuff that we want to use that's still being debated by the Rust team, and therefore marked unstable. As each release goes on, they stabilize these APIs, but it's sometimes a pain working around useful APIs that are marked unstable just because the dust hasn't settled yet within the core team. Having said that, we totally understand that they're being thoughtful about all this, because backwards compatibility implies a very serious long-term commitment to these decisions.
> On the improvements side, we're in close contact with the Rust core team
I've read before (somewhere, I think) that Dropbox effectively maintains a large internal "standard library" rather than relying on external open source efforts. How much does Magic Pocket rely on Rust's standard library and the crates.io ecosystem? Could you elaborate on how you ended up going in whichever direction you chose with regards to third-party open source code?
So, we use quite a few crates for the things it makes no sense to specialize in Dropbox-specific ways.
>Are you going to open source anything?
>Probably. We have an in-house futures-based I/O framework. We're going to collaborate with the rust team and Carl Lerche to see if there's something there we can clean up and contribute.
[0]: https://www.reddit.com/r/rust/comments/4adabk/the_epic_story...
We also tweak jemalloc pretty heavily for our workload.
1. Go vs Rust at Dropbox's scale and requirements.
2. Maintaining availability whilst moving from AWS to Diskotech and Magic Pocket.
3. Internals of Magic Pocket (file-system, storage engine, availability guarantees, scaling up and scaling out, compression details, load balancing, cloning etc)
4. Improvements in perf, stability, security, and cost.
Thanks.
Will most likely start with the following: 1. Overall architecture and internals. 2. Verification and projection mechanisms, how we keep data safe. 3. How we manage a large amount of hardware (hundreds of thousands of disks) with a small team, how we automatically deal with alerts etc. 4. A deep-dive into the Diskotech architecture and how we changed our storage layer to support SMR disks.
Hopefully these will be of a sufficient level of detail. We certainly won't be getting into any details on cost but we're pretty open from a technical perspective.
(Lemme just take a moment to say how great Backblaze's tech blog posts are btw.)
One of the ways tech companies wrong is by putting the tech first and the users second. I think Dropbox's early competitive advantage was putting the user experience first. Everybody else had weird, complex systems; you folks just had a magic folder that was always correct. User focus like that is easier to pull off when it's just a few people. But once you break the customer problem up into enough pieces that hundreds of engineers can get involved, sometimes the tail starts wagging the dog and technically-oriented decisions start harming the user experience.
How did you folks avoid that?
Here's a request, how did you migrate all that data. Are you going to be open-sourcing any of the tools that you built up? We'd be interesting in hearing about that process. A lot of folks using B2 right now are trying to do this exact thing (or making copies if not actual migrations). Cheers! ;-)
Cool stuff, really interesting read!
But now that you've moved away from AWS, have you expanded your security team to help make up for the fact that you need folks to cover all your infrastructure security needs now too? Have you made the hires you needed to deal with the low level security issues in hw and kernel? How have you all been dealing with this so far?
I ask, because I help people make sense of their security issues and my clients explicitly ask me about your company. So keeping track of what you all are doing there is pretty relevant!
Feel free to refer others to this thread if you think it'd be helpful.
The data placement isn't RAID. We encode aggregated extents of data in "volumes" that are placed on a random set of storage nodes (with sufficient physical diversity and various other constraints). Each storage node might hold a few thousand volumes, but the placement for each volume is independent of the others on the disk. If one disk fails we can thus reconstruct those volumes from hundreds of other disks simultaneously, unlike in RAID where you'd be limited in IOPS and network bandwidth to a fixed set of disks in the RAID array.
That probably sounded confusing but it's a good topic for a blog post once we get around to it.
http://msr-waypoint.com/en-us/um/people/yekhanin/Papers/Usen...
It's only covering a few use cases and doesn't go into too much detail and shows example code but it seems like using TLA+ has been beneficial for them.
You must have left a large hole in AWS' revenue stream. I assume their fees for transferring your data out of their system helped cover that short-term, but 500PB of data is a lot of storage capacity to sell.
Your other answers indicate your separation with AWS was very amicable, much more amicable than I would have expected. Either "its just business" or they have plans to backfill the hole you left.
So 500PB is ~6% of that... Although I'm sure Amazon would have liked to keep Dropbox as a customer it's probably a pretty small percentage of their revenue. Also, if you assume that AWS is growing at ~50% year over year then 6% of S3 is only about one or two months worth of growth.
I think it was this talk but I'm not sure: https://www.youtube.com/watch?v=3HDQsW_r1DM
I'm surprised at this. I would have expected the opposite: using AWS for the application and metadata but managing your own storage infrastructure. Much like how Netflix runs on AWS but streams video through its self-managed CDN. Can you shed some light on why it was designed this way?
Our previous version of the storage layer ran on top of XFS which was probably a bad choice since we ran into a few XFS bugs along the way, but nothing serious.
On a high level it's variable-sized blocks packed into 1GB extents, which are then aggregated into volumes and erasure coded across a set of disks on different machines/racks/rows/etc. We also replicate cross-country in addition to this. Live writes are written into non-erasure-coded volumes and encoded in the background.
The volume metadata on the disks contains enough information to be self-describing (as a safety precaution), but we also have a two-level index that maps blocks to volumes and volumes to disks.
More info to come later.
Also, do you mimic the eventually consistent behaviour of AWS, or do you offer a stronger form of consistency?
"The irony is that in fleeing the cloud, Dropbox is showing why the cloud is so powerful. It too is building infrastructure so that others don’t have to. It too is, well, a cloud company."
Wait ... so using AWS is "cloud", having your own servers is "cloud" too. Everything is cloudy!
I hope there will be no "IoT over IP"
I've never been more confused in my life. "Wait, what does this version of Windows do?"
Luckily they rebranded it as just "Microsoft Azure" which makes a shit ton more sense.
The second aspect is paying on an incremental basis for someone else to provide the pools developed in the first part. But that's more or less a trivial extension once you've got the first.
I count 6 columns of 15 rows/drives. Might be 7 columns even, there's some panels that aren't fully open on the video.
So, 90-105 drives. I'm guessing they are using 10TB drives, although maybe they can get bigger unannounced drives? Roughly, the math seems to check out.
Quite impressive. Guess the Backblaze guys need a Storage Pod 6.0 soon :P (I know, I know, different requirements/constraints)
Yea :D Though if we used 10TB drives we could get about 450TB in one pod, it would just be hilariously expensive for us to do so. Granted that's not 1PB per pod, but we'd imagine our costs are lower. But yes, different use-cases, and we're working on 6.0 ;-)
It's an interesting subject. I'll never click on or be persuaded by ads, so they're not really gaining anything by showing me ads. I'm essentially worthless traffic to them no matter how they cut it.
I do understand the problem they're trying to solve, and I would like to pay for content that I find worthwhile. A convenient microtransaction protocol where I _don't have to sign in or interact in any way_ would be nice. (I don't want to waste time interacting with their bespoke account system / sign-on, even if it is "frictionless".)
Also, Google Contributor network let's you block any DoubleClick ad, while still paying the publisher for the page view, so that might get what you want.
Doesn't matter, impressions are valuable to marketers as well.
"or be persuaded by ads"
Yes you will, you just think you don't/won't.
That's a really blanket statement. I'm rather minimalist, and I typically form negative opinions of the things I see in ads.
I saw a comment about the north korea pics earlier yesterday. It was referring to propaganda. Apparently the purpose of censored speech and propaganda wasnt so much to sway opinion and be cloaked as truth, rather it was an irresistable pleasure for rebels to decry and thereby reveal themselves. You might think that a deliberate attitude toward advertising can defeat it but the subconscious does not answer to your control :-)
Sure, until it's something that fits your world view, and then you don't.
Look, I'm usually the first to (excessively) qualify sweeping generalizations I make with 'generally', 'statistically speaking', 'ceteris paribus' and other weasel words. Yet in the context of pervasive marketing, I don't need to - everybody is swayed by some form of advertising, whether you realize it or want to admit it or not; 50 years of research and billions of dollars spend on that research tell us so unequivocally. But if you don't, it's better you start admitting it to yourself - if only because it gives you better insight into your own internal psychological processes.
Then again who am I to give unsolicited paternalizing advice to strangers.
According to disconnect that one page sends data to 15 different advertising sites, 15 different analytics services and many other content services. So much for fighting for their readers privacy online.
https://blogs.dropbox.com/tech/2016/03/magic-pocket-infrastr...
- Python is our primary development language for most stuff at Dropbox.
- Mobile development and UI code use whatever is appropriate for each platform.
- Go is the primary language for Infrastructure, meaning fairly deep-backend stuff: databases, storage systems, message pipelines, search indices etc.
- Rust is used on Magic Pocket (which is within Infrastructure).
- We still have a bit of C++ code floating around but there's not much new development in C++ within Infrastructure. C/C++ are still used for common libraries for mobile development however.
Dropbox stores quite a bit more data than Netflix. Data storage is our business. Ergo, we control storage hardware.
On the other hand, Netflix pushes quite a bit more bandwidth than almost everyone, including us. Ergo, Netflix tightly manages their CDN. That's where they really focus their technical work. Their storage footprint is smaller and less important to optimize to the extreme.
Really glad to see what's been going on from behind the scenes, and I'm looking forward to seeing what the future will bring to the front end of Dropbox.
Thanks guys!
As usual, we actually have more challenges pulling the libraries forward than the language.
90x 3.5in disks, 2x compute nodes with 2x E5-2600v3 CPUs.
Can't identify the 3rd one yet...
EDIT: Found it... HPE Cloudline CL5200, 80 drive bays. http://www.nextplatform.com/wp-content/uploads/2016/03/hpe-c...
46GB/s, is my math right?
In terms of your customers getting data to you, again, fiber interconnects inside the DC are your best bet. Colo'ing is the fastest way to move bits because it relies on arteries, not capillaries, for moving bits from place to place (no last mile problem).
Did you guys get into custom network hardware or still a Juniper/Cisco shop?
A lot of the system design is about talking closely to vendors about MTBF for specific loads, and we use a lot of metric data to characterize our loads accurately in those calculations. Then, of course, we measure once we're in the field to make sure we were on target, so the next round of estimations is better.
> So, in the middle of this two-and-half-year project, they switched to Rust on the Diskotech machines. And that’s what Dropbox is now pushing into its data centers.
See https://www.reddit.com/r/rust/comments/4adabk/the_epic_story... for more information.
1. Dropbox moved from AWS to its own datacenters after 8 months of rigourous testing. They didn't exactly build a S3 clone, but something tailored to their needs, they named it Magic Pocket.
2. Dropbox still uses AWS for its European customers.
3. Dropbox hired a bunch of engineers from Facebook to build its own hardware heavily customised for data-storage and IOPS (naturally) viz. Diskotech. Some 8 Diskotech servers can store everything that humanity has ever written down.
4. Dropbox rewrote Magic Pocket in Golang, and then rewrote it again in Rust, to fit on their custom built machines.
5. No word on perf improvements, cost savings, stability, total number of servers, amount of data stored, or how the data was moved. (Edit: Dropbox has a blog post up: https://blogs.dropbox.com/tech/2016/03/magic-pocket-infrastr... )
6. Reminds people of Zynga... They did the same, and when the business plummeted, they went back to AWS.
7. Not a political move (in response to AWS' WorkDocs or CloudDrive), but purely an engineering one: Google and then Facebook succeeded by building their own data centers.
Actually, full disclosure, we really just rewrote a couple of components in Rust. Most of Magic Pocket (the distributed storage system) is still written in golang.
> No word on perf improvements, cost savings, stability, total number of servers, amount of data stored, or how the data was moved.
Performance is 3-5x better at tail latencies. Cost savings is.. dramatic. I can't be more specific there. Stability? S3 is very reliable, Magic Pocket is very reliable. I don't know if we can claim to have exceeded anything there yet, just because the project is so young, and S3s track record is long. But so far so good. Size? Exabytes of raw storage. Migration? Moving the data online was very tricky! Maybe we'll write a tech blog post at some point in the future about the migration.
We're always about the right tool for the job tho, and there are definitely use cases where Rust makes a lot of sense. We've been really happy with it so far.
In fact, just think of Rust as C++ for mortals. :)
At a high level it's really a memory thing. There are a lot of great things about Rust but we're also mostly happy with Go's performance. Rust gave us some really big reductions in memory consumption (-> cheaper hardware) and was a good fit for our storage nodes where we're right down in the stack talking to the hardware.
Most of the storage system is written in Go and the only two components currently implemented in Rust are the code that runs on the storage boxes (we call this the OSD - Object Storage Device) and the "volume manager" processes which are the daemons that handle erasure coding for us and bulk data transfers. These are big components tho.
But our web controller code, for example, is millions of lines of Python. And we use Python in lots of other places as well.
I think both durability numbers would be fine for customers, but I've also wondered about the math behind AWS S3 durability for a while, and how you would prove statistically that 11 9's was even possible.
> Dropbox still uses AWS for its European customers.
We haven't publicly launched EU storage yet but will be doing so later in the year.
> Dropbox hired a bunch of engineers from Facebook to build its own hardware heavily customised for data-storage and IOPS (naturally)
Facebook and Google and startup folks and people from random other places.
Our IOPS demands are reasonably modest on a per-disk basis which is why we skew heavily towards storage density. This is a different workload than you'd find at some of the other big players, hence a fairly custom solution.
> Not a political move
Definitely. AWS are great, we love working with them and will continue to do so.
Also, was the S3 "infrequent access" tier a response to customers like you or was your special bulk pricing already taking into account your low IOPS demands?
Thanks!
If Box is in the "Business Services" business, then perhaps it's different.
Dropbox also had that email app didn't they that they recently announced was closing down - mailbox if I recall correctly.
Can anyone convince me that this move by Dropbox isn't going to end very badly for them?
EDIT: downvoters - is HN unable to have a debate about whether this was a smart move or not.
We've had our own datacenters for a very long time; this project just represents moving "blob storage" into our own datacenter rather than just metadata.
> Mass storage with Amazon is much cheaper
As the article says, in Dropbox's particular case this is very untrue. Although I agree with your general point that, absent having something like Dropbox's scale/resources, just using S3 is probably your best bet.
> Can anyone convince me that this move by Dropbox isn't going to end very badly for them?
So, wrt "ending badly", the project has already happened. It's more efficient, faster, more flexible, etc. So I can say with some confidence it won't end badly. :-)
Also I am sure the in house data centre looks cheaper to Dropbox - but now they have their own hardware on the balance sheet. But I meant cheaper from a consumer point of view - I get unlimited storage at Amazon for $60 a year.
i don't know if this is true or not, but it isn't at all necessary.
you can just buy everything on an operating lease for a fraction of what amazon or any other provider charges and write the payment off without any sort of depreciation schedule or hit to your cash or unnecessarily grow the balance sheet/liabilities.
how do you think it's possible that you can easily get a $15/month dedicated server?
Sorry makes no sense to me.
Even taking into account depreciation, administration, maintenance etc, if you have a large enough consistent need amazon is not the lowest cost option by a significant margin. This isn't a controversial statement.
Also, presumably, not all the AWS users are awake at the same time ;-)
yeah, they sure do.
The cloud providers are in a cut throat business (mitigated somewhat by value added services), so the price will bob along near cost (though they may eak out more profit on some work loads compared to others).
These costs are known and planned for. Dropbox wouldn't do this without running the numbers through vast herds of accountants.
Now here is the kicker. Amazon does this too (but slightly less due to economies of scale) and then makes sure their billing covers this.
Your main argument seems to be that Amazon, Microsoft etc will undercut Dropbox on price which is fair enough. However Dropbox has done well so far offering better service and I can see that continuing. The better service thing is mostly about the people running the services and Dropbox seem good.
To see how they are moving away from commodity storage check this out:
("Dropbox used to be fundamentally about storage but now it is changing how we view productivity and collaboration") https://www.siliconrepublic.com/enterprise/2016/03/11/dropbo...
There's an infection point somewhere at which owning your own hardware and paying someone to acquire, build, manage, and maintain it becomes cheaper than renting from someone who does the same thing at scale and makes a profit off of it.
Dropbox is probably there, but many small web apps are not.
Suppose you are on S3 and you can serve a customer x Mb/$. Now you're limited to products that you can charge more for. Suppose you get data served to customers on dedicated at x/10 Mb/$ (easily doable, and compared to S3 I bet x/50 would be doable).
Isn't this how products like facebook video, vimeo (and thousands of porn sites) survive ? If you calculate how much it would cost to serve 20 Mb out from EC2, you would never ever be able to pay that using advertising revenue, even if you got as much as TV gets.
So that reduction in price that comes from not using the cloud doesn't just go into your pocket : it enables new business models that just aren't accessible to you otherwise.
Also can we stop pretending that the alternative to the cloud is buying servers from HP and colocate them ? That's simply not true. There's any number of projects that would enable you to slowly scale up that aren't EC2. It's more work, certainly. But it's worth it at quite small scales.
And in some ways (e.g. geographic reach) clouds simply don't match existing dedicated offerings. Not now, and at a 20% yearly price reduction it'll take them decades to match dedicated server rental.
Obviously I'm not claiming clouds don't have advantages. But you're paying quite a high price for them atm.
Currently, S3 will provide bandwidth out for 300TB/month at ~$21,000.
At our web host the same bandwidth would cost $1800.
At our CDN, the same bandwidth would cost $3585 to $4791 depending on the number of points of presence you'd require (6 vs 17).
S3 and its extreme availability profile is nice, but its not cost effective when the difference between pricing vs the lowest-priced option is almost $20,000 a month.
To me, the main alure of S3 was never amazon's architecture or HA, but that you no longer had to manage your storage servers and add capacity in a stepwise process. 7 years ago, to do this you were stuck with GlusterFS or even MongoFS.
Today we have OpenStack, Ceph, RiakCS and others providing battle-tested open-source solutions that anyone can run so there's much less of a reason to go S3.
I like to compare Amazon to Akamai with the statement of "yes, they're the best or close to it; but do we need the best?"
It is sad that bandwidth charges out are so expensive because it does make one question their services.
S3 raw price-per-gig is somewhat competitive with what Dropbox built, but the ancillary things around it are not remotely competitive.
That doesn't matter, they have gained a good foothold in SMEs which is where the real money is.
> Dropbox also had that email app
That was a peculiar acquisition, I guess more of a acquihire for a team that made some waves in UI design. They would have folded anyway, Google has caught up with the "swiping todo" concept.
> Can anyone convince me that this move by Dropbox isn't going to end very badly for them?
A company on that size has many metrics they can use to define success. Owning your metal is a big investment, but it also brings a level of control and performance that you will never get from outsourced infrastructure. Google became Google without AWS ever entering the picture, after all.
I think these are the first moves in a wave of "de-cloudification" by the fields' own pioneers. The longer you've been on the cloud, the more you're intimately aware of its limits and its costs. No technology is a silver bullet.
Dropbox is probably trying to position themselves as a secure data storage solution for larger organizations. Being on Amazon scares away a lot of people. I'm sure we'll hear about some new paid services from them which will be compliant with all manner of regulations. This will open them up to huge multi-million dollar contracts and they'll get baked into some big companies for years to come.
Isn't that Box's business plan?
I don't think the people scared by Amazon would not be scared by a company with Condoleeza Rice on the board. Amazon is already compliant with all those regulations you speak of -- they host large swaths of USGov infrastructure, after all.
More likely, DB people have simply come to the realization that, at their scale, AWS is extremely expensive for what it delivers. Being a technology company, they are not scared by the thought of building and maintaining their own iron in order to make substantial savings. Their core business is storage, after all.
> I imagine a lot of people are like me i.e. Store most of the stuff at the cheapest location and use Dropbox just for docs that you want to sync on multiple devices
I don't believe that's a correct intuition. You're describing optimizer behavior, but for most things people are saticficers [1].
I happily pay Dropbox $100 a year to make a problem go away. Is it the best deal? I don't care. I'm not going to mess around over $0.27 per day. For me it's magic syncing and backup for all my important stuff. Using Dropbox may not be cash-optimal, as I'm currently paying something like $4 per GB/year. But for this I'm not an optimizer, I'm a satisficer, and as long as Dropbox maintains my experience of perfect reliability, I'll keep paying them.
Most of the world is not HN readers, it is McDonalds workers (gross generalisation for affect, not accuracy). Dropbox is the greatest tool ever when dealing with the vast majority of workers, because it just WORKS, without a lot of explanation. Work in this folder always or get written up. DropBox is still my favourite SAAS service, and every time a co-worker loses a file I just want to punch them.
> downvoters - is HN unable to have a debate about whether this was a smart move or not
I doubt the downvotes are in opposition to discussing it. I think it's your poor framing of it. You basically say, "these guys are doomed because I imagine things". It's not other people's job to argue you out of your errors.
If you'd actually like to start a discussion, try saying something like, "I don't understand why X" or "How will Dropbox overcome factor Y". When you come out swinging, you start an oppositional dynamic, and downvotes are a common response to an aggressive opening that isn't so compellingly put that people feel it's worth responding to.
Buying off the shelf hardware providing an s3 style API saves us a significant amount of money, and while our scale is large, it isn't a fraction of what Dropbox is.
If you're not going to adopt wholesale, the outbound network costs make it very expensive.
You're missing something here, which is that DropBox are probably now spending roughly 10-20% of what they were paying Amazon on an annual basis. The savings probably grow as DropBox grows. Those savings over your hypothetical 5 years could probably keep DropBox on life support for another 5 years.
I really doubt their in-house storage backend will be cheaper compared to the lowest rate they can get from one of the cloud providers.
AWS marginal cost increase for supporting Dropbox is far below the cost of Dropbox building and maintaining their own solution.
Given these facts they should be able to negotiate a far better price/cost than their current solution of building it in-house.
There is nothing unique about their requirements which justify this move. It's a really weird decision. Probably we're missing something here. This must be part of some larger strategy they are executing.
The cloud is for stuff you don't care too much about.