The Meteoric Rise of DigitalOcean
news.netcraft.com
news.netcraft.com
It's that the web interface, and process for starting a "droplet", is dead-simple and super-elegant, their support docs are comprehensive and easy to use, and they even do things like let you manage your own DNS records. And it all just works.
Digital Ocean is 100x easier and quicker to use than AWS, without exaggeration. It's easy to start a hosting provider -- it's really hard to make one that is also really high quality, intuitive and easy to use, and then cheap on top of it.
I've used Digital Ocean for a couple of years for small personal things, I'm actually currently in the process of moving a bigger site from AWS to Digital Ocean, at this point it's turning into a no-brainer.
100% agree. I really think the markets for DO and AWS are different. DO aims at the "I need a few boxes" sort of user, and AWS aims at the "I boxes and other infrastructure" where other infrastructure is databases, auto-scaling services, etc.
DO feels like Linode felt like four years ago: just the basics at a great price point. (I still like Linode, but they are probably competing with both AWS and DO, which is a tough spot to be in).
This is the typical pattern for market disruption: competitor enters the market at the low-end and moves up-market to eventually dislodge the incumbent.
We spent most of last year dealing with the challenges of scale as we grow quite a bit.
There was the need to scale infrastructure, engineering, customer support, as well as the misc odds and ends of running a business (office space, etc.).
As we grew from 5 people to over 50 today that definitely slowed us down on our product roadmap substantially but now that we've scaled out most teams (and we're still hiring! =]) we are able to once again refocus our efforts on engineering.
That means more updates to the backend for stability and also rolling out new features. During the next couple of months customers should see the benefits of those efforts.
There are other challenges that come from scaling rapidly including making sure that we can retain our culture as new team members join. Aside from rolling out new features and diving into some of those we'll also be writing blog posts on the scale challenges we faced as a startup so that it hopefully provides some insights to other startups as they go through their growth phases as well.
As always if there are any questions please reach out to me direct - Moisey -- digitalocean.com - It may take me a day or two to respond depending on how much work is piled on top, but I always read every email and get back to everyone and we very much appreciate the feedback.
Thanks, Moisey - cofounder digitalocean
AWS vs DO seems more like Ford vs Toyota than cars vs horses.
> An example that has nothing to do with “high tech” comes from the mechanical excavator industry. This industry was dominated by steam shovels until the 1920’s, when gasoline powered engines began to replace them. This was, however, not a disruptive innovation, but a sustaining one, even though the design of the machines changed radically from that of a steam-powered system of cables, to that of a gasoline engine driving a system to extend and retract the cable connected to the bucket. The new engines were more capable than the old ones, and were better at doing more work more reliably, and cheaper than the old system. Despite the radical change in the industry, the same firms that were strongest in steam shovels stayed on top. The disruptive change came with the introduction of hydraulic-actuated systems after World War II - a change that eliminated nearly all of the established players by about 1970, in favor of companies that entered the market with hydraulics. The first hydraulic-based excavators were less capable than the cable systems that were in existence, and certainly couldn’t compete with them. However, they were small enough that they could be deployed for jobs previously done by hand, opening up a new market, in which the desired attributes were quite different from the big jobs that the cable actuated excavators were used for. The technology involved in hydraulics continued to improve, however, and with time eventually equaled and then surpassed the needs formerly filled by cable-based systems. In the meantime, though, the established firms were still going strong, and didn’t do much, if anything, to deal with the new competitor (because it wasn’t really seen as a competitor, not being sufficient for their existing clients’ demands) until the new arrivals were “in the midst of their mainstream market”. By the time the established companies introduced their own hydraulics, however, it was too late, and the later entrants were by then better positioned with the new technology.
From http://www.squeezedbooks.com/articles/the-innovators-dilemma... - a site I sold on a few years ago.
Holy crap is AWS bizarre to set up. I got it done without too much trouble in the end, but man, the UI is just atrocious. It took way too long just to find the right place to go to get started with the process.
phpMyAdmin is great when you're first learning to work with the LAMP stack; it lets you fiddle with values, manually create databases, munge columns, etc.
But.
If there are multiple instances of your system running "in the wild" (even just within your own company), and their versions will ever have even the slightest chance of getting out-of-sync? You really want the entire change-history of your modifications to the database checked into git. You want to be able to take any previous sorta-known database state, and move it toward the current well-known database state. To do that, you need programmatic migrations.
And as soon as your software ships a "db/migrate" folder, (non-readonly) phpMyAdmin access becomes anathema to proper configuration management.
AWS can be thought of as a big database that contains your EC2 instances and snapshots, your S3 buckets, etc. At scale, you only want to interact with this database programmatically.
So: if this was the correct hypothesis for why the GUI sucks, what would be an expected prediction? That the programmatic APIs for manipulating AWS would be great.
And they are! All the HTTP APIs for AWS--the ones you would use in any automated provisioning script--are simple, clear, and pain-free. They're definitely the "first-class" road to AWS, documented to heck, and it's clear that Amazon itself dogfoods them directly.
But I think there are plenty of mid-sized comapnies with multiple AWS accounts, with many machines in each one where no one is quite sure what they do or if they can be switched off.
Disclaimer: Sysadmin/Devops
So how does DO's api compares to EC2?
Meanwhile AWS has this nasty outdated SOAP API that you don't want to touch by hand with a 10-foot pole, you had better have a good AWS library for your existing language of choice or know how to make sense of a WSDL.
Um, no. Every AWS service has a REST API. The only difficult thing is calculating authentication headers, but that's nitpicky for a good reason.
S3's API is extremely RESTful -- not perfect, but really good. Some of Amazon's newer APIs aren't as RESTful, but they're usually still better than what many people call "REST" today (i.e. "Hey it's not SOAP, so who cares if every flipping thing is a GET request, it must be REST!").
Deleted comment
Yeah, AWS has some functionality/scaling features that DO and Linode don't, but it's a smaller set of functionality than I think some people initially assume.
EC2 has two types of storage: local disk "ephemeral/instance storage" and network storage "EBS". Digital Ocean only provides local disk. In EC2 you can mix and match, but EBS is the most convenient.
If you do a stop and start on EC2 (very different to picking "reboot"). You will lose what's on local disk because stop/start is basically requested that you be moved to a different physical box. Data stored on EBS is not affected. On Digital Ocean if you "power off" you remain on the same host, but get billed at full rate whereas on EC2 you don't get billed for the time it's stopped. Rebooting on EC2 is the same as rebooting on DO.
If you use EBS for storing everything your data survives even if the hardware your instance is running on breaks. If the hardware breaks on Digital Ocean or on EC2 if you only use ephemeral (local) storage you lose your data. It's the same story on DO as it is on EC2. The only difference is that EC2 gives you the option of storage which can survive the physical box you are on dying (EBS).
Is that really the case - neither are using a SAN or RAID or other backup method to ensure data is preserved in (almost) all circumstances? Not so bothered about EC2 (due to EBS) but for Digital Ocean..? Poor show if that's the case.
Some examples:
* DNS can be set up to round robin * auto-scaling can be built in * load balancers are available and easy to use * For companies with multiple employees various levels of access can be granted to the system via IAM roles * you can add and subtract hard drives from an instance on the fly (EBS volumes) * You can set those volumes up as RAID if you so desire. * You can easily move IP addresses from one instance to another.
These are just the ones off the top of my head as I've been off AWS for about 6 months now. This doesn't even count how well the servers play with other amazon web services, like SQS, SNS, SES and RDS (queue, notifications, email, database).
That being said, I use DO for a few small sites I host because I don't need any of the features AWS provides.
I'd think round robin DNS has nothing inherently to do with the infrastructure - that's a function of the DNS server, right?
Yes, there are some things AWS has that other services don't. My own experiences have been that people default to thinking AWS==cloud and that's the 'best' option, and this thinking was prevalent years ago, well before SES, SQS, RDS, etc were extra reasons for integration. They've got good services, but have done a hell of a branding job over the years as well.
Deleted comment
I couldn't possibly care less about an admin panel. I'm a Linux guy.
I want cheap speed and reliability. With DO, you get all of them. Sorry Linode, but you're old news now.
I have a server in each and I like them both. However;
- Linode support is second to none. DO is not bad either but much slower and less responsive. You feel the difference when you really need it.
- I had a CC payment issue with DO since the beginning. I mentioned it to the support twice but unsolved as of yet, using paypal.
- You can't assign multiple IPs to your instance in DO. I have 6 IPs in one of my Linodes for SSL reasons. This is more than enough to compensate the price diff, actually Linode is cheaper because of that.
- No built in, easy-to-use load balancer in DO.
- Networking feels much faster and stable on Linode, at all occasions.
- Small things like "no more Amsterdam based droplets due to IP restrictions" when you urgently need a server don't usually happen on Linode.
Things like these pile up in the long run and you can easily justify the price difference. If have a very serious/critical project I'd definitly chose Linode over DO.
Do you not know about their major security incidents ?
http://venturebeat.com/2013/12/30/iaas-provider-digitalocean...
So for me, on average a $20 Linode has less RAM but performs better for CPU bound tasks than a $20 DO Droplet.
Yes, I've even used them several times on non-DO servers because they're quite good and rank high on Google.
I find Digital Ocean's community/blog content impressive. In most instances I can google "Install X on Digital Ocean" and they have a solid, well updated walkthrough.
AWS does have some cheap offerings - some micro instances for your first year are free, which is less than "$5/month, good for tinkering", but for the most part they're not interested in folks who have only a couple bottom-end boxes, because that market doesn't have a lot of money in it.
I think much of this thread is a testament to the lower value customers being ignored. Things about how DO is much less complex, simpler and as a result cheaper.
Here's part of a great talk from Christensen describing the same logic but in the Steel industry: http://www.youtube.com/watch?v=B5FxFfymI4g
The problem with the 'logical conclusion' that you're offering is that it works entirely in a vacuum. It assumes that AWS does not innovate itself, but steams on with a stolid product. Nothing could be further from the truth - barely a week goes by without them announcing a new service or improvements to an existing one. DO is behind the 8-ball-that-is-AWS, but AWS isn't just sitting there on the felt, it's moving too.
Given the culture of innovation that exists at AWS, it's not hard to see how they could adjust themselves if DO actually does become a threat. AWS is not head-in-the-sand Microsoft, and even if it were, the might of all other operating systems combined - including that of the company so successful that it has $100 billion just lying around in cash - still hasn't knocked Windows off the top spot. Hardly the 'downfall of the incumbent' suggested.
And Microsoft is still undisputed king in the realm of enterprise user software. Unix has always shared the server room with MS (actually, MS is the interloper there), but outside the server room, enterprise users live on MS. There is a myriad of cheaper enterprise and/or office software out there, and MS isn't particularly threatened by it.
Things about how DO is much less complex, simpler and as a result cheaper.
And then when you need the big stuff... the stuff for which you need to pay the big bucks... the complexity isn't there and you move to the vendor that does supply it. When DO makes those kinds of offerings, then AWS will see them as a serious competitor, and you will see prices drop sharply. Until then, it keeps on truckin'.
When they added cloudfront, RDS or elastic beanstalk they were offering "something less flexible but very simple" even while they were adding glacier or MFA through IAM which are for more "complex" customers.
I think I can see where you're coming from but I strongly disagree that AWS will protect its market share by adding more products.
Each new product doesn't just bring utility in terms of what customers can do - it also brings disutility in other areas. New products dilute marketing (what are you selling?). It dilutes the allocation of internal resources (maintenance, R&D, marketing). It increases cognitive overhead for customers (E.g. the product range is a mess -EC2/S3/SWF/SQS/SES/SNS/FPS How is a customer to keep track of what's going on?).
My assertion is that for the average customer - new AWS products only bring headache and zero benefit. Because most people just won't find the need for it. Also AWS don't need to care. They will not meaningfully improve the core offering to the average user because it's the high end customers that will move the needle at the end of the day.
Bullshit. The real reason is it is MUCH, MUCH cheaper.
And it works. That's the reason I use it. I am an honest person though, so most people probably won't share the same viewpoint.
The extremely low barrier to entry isn't just price point. It's hard to say which is more important, but the simplicity difference between DO and competitors is greater than the price difference.
The real reason is (obviously) a combination of cost and simplicity.
With D.O. there's an ambiguous list of things you need to do to upgrade to a new plan. For example, after shutting down to upgrade, I heard you need to backup your disk image to multiple datacenters because sometimes an image can get deleted by accident. One night I spent several hours trying to (safely) upgrade and support was slow to respond, eventually I gave up. With Linode it's just a few clicks and you can see the progress of the migration in the dashboard, it's very easy.
I use both Linode and D.O. but for personal projects, Linode is my choice because their tech support team is very responsive 24/7. I am not a heavy user of tech support but when I do need help I need it fast!
So unless your data is worthless then Linode is a terrible choice.
Fortunately, my data's not that important.
If I cannot have freedom of speech on my blog, I will have to find a new provider that respects my freedom.
Only downside is that they're a bit limited in what kinds of platforms they support.
Did you read the article in question? http://vpsexperience.wordpress.com/2014/01/05/googler-speaks...
It took four comments, out of context, and made it into a slam. Bias of the rant aside, DO says that it was because of a TOS violation on privacy and defamation. It comes pretty close to outing where the person lives, what websites they run, and certainly makes plenty of assumptions about the four comments.
It reads like someone bitter lost an internet argument when they couldn't bait the employee into saying something incriminating, and they wanted revenge.
Exactly this. This guy baited someone to say something about his employer, posted quotes out of context and with obvious bias. The guy seems like an asshole.
Normally I would be right there with you - but in this case it was a troll who was asked to anonymize his clearly defamatory attacks, and predictably he blew up over it.
DO is a private provider, not government affiliated, so 1st amendment doesn't apply and they are protected under §230, but still they have a TOS that prevents it from becoming a safe haven for cyber bullies like this troll. TBH I'm okay with this, especially since the troll agreed to the TOS to begin with.
There may be small risk of this scenario happening, but if you are comparing to another host that doesn't make service contingent upon policing of content you host, then any risk is infinitely larger than necessary. In general, minimizing the ways my business could be wiped out by some service provider helps me sleep better at night. If a user uses a service I provide to host something defamatory, I want there to be a court order before the web host goes flipping power switches on my racks, as it's likely I'll have been contacted by someone before it gets to that point.
Let's start by saying that DO has fewer features for a complex deployment:
No load balancing,
no private subnets,
(IIRC) no IPv6 at the moment.
No "cloud files" or S3 equivalent,
the scaling isn't as automatic as RAX or AWS can provide.
DO can't host Windows (which still matters for many existing operations, don't fool yourself).
RAX gives more information and granularity on its VM levels.
Finally, support. RAX can do managed services, dedicated advisors, IT support and servers for sensitive data. DO has none of this.
---
There are two things DO does that I appreciate (though neither concept is new):
* $5 hosting. RAX is a minimum of $16/mo now, for the same VM strength.
* Application deployments. The ability to one-click deploy Wordpress/Gitlab/etc. is pretty damned awesome.
DO is great for technically competent people who don't need a lot of infrastructure, or otherwise have a consistent level of server/bandwidth requirements for their servers.
I imagine the change was a lot like going from DO to AWS - much more complicated, but at the time Rackspace had only owned Slicehost for a few years and the interface was awful and the setup was confusing with 1st gen and 2nd gen cloud servers and it was just a mess.
Also the VPS up-time we have experienced on over 20 nodes for the past 3 years on Rackspace cloud has been pretty great, all things considered.
As far as I can tell it's 5 times the cost of digital ocean.
You know you need it, so why pay a huggggeeeeeee premium and get relatively poor performance compared to dedicated for on demand VM services?
I get the impression a significantly smaller subset of people using AWS/Rackspace/Azure should actually be using them.
Obviously I know nothing about what you're doing, so it may be that you have a perfectly good use case.
I don't see this on their site (http://www.rackspace.com/cloud/servers/pricing/). Minimum pricing there is $29.20/mo for a 1gb instance.
Actually Rackspace does have a deployments option under servers that can deploy Wordpress, GitLab, and others. It even handles some of the configuration details right out of the box.
You should start with this. How are they even comparable if one is more than thrice the price of the other?
To me they are running a pretty transparent pump and dump scheme. Promise a great service, sell it at a loss, get a ton of users, do a shit job of supporting it, take on investment money, cash out early, profit.
I think they got really popular suddenly and couldn't handle the explosive growth.
I am now nervous about moving to them.
On the production server, I do ping the server every five minutes, but the customer complaints always reach me before that system.
The Good: - easy setup via the Web UI - instances provision fast - They had a 960GB instance type for only $960/mo
The Bad: - Support was lackluster, kind of like "stuff happens, so your servers may or may not keep working" - DDOS mitigation was overly aggressive. A DDOS likely targeted at the prior owner of the IP caused DO to take the box offline for two days with no warning or even a note saying it had been done. It took ~30 hours for them to even figure out that they had done this. - No SAN storage, so no practical way to store data in excess of what the largest instance type will hold. This limits the utility of the largest instance type. - The killer: they have no capacity of 96GB droplets, so if our box died, we would be unable to restore from the DO snapshot. This ultimately caused us to migrate off of DO (to AWS).
So I still run a couple of nonessential projects there (total spend < $20/mo), but I wouldn't trust my paycheck to services run at DO just yet.
So do most blog posts reviewing DO out there as well. Give it another shot, or document exactly what you did and I'm sure someone will figure out the issue.
http://trends.netcraft.com/www.digitalocean.com
We have been publicly tracking DO at that address on a monthly basis (I work for Netcraft).
(University of Bath CS grad here.)
They do have higher support costs, credit card transaction fees, and ip address costs. They would probably lose money if the primary users of the $5/month plans were doing punishing things like video encoding, but my guess is the vast majority are no load users.
I once read someone that was doing hosting, said the larger plans were actually more challenging. 4 high volume forum sites on the same host would cause IO contention. Digital Ocean has SSDs and they mix and match sizes on the physical host, so it should be less of a problem for them.
Linode does actually almost fit the bill, except that they require manual authorization for an additional IPv4 addresses. So there's no way to script deployment of such a system.
I have gone through about 10 VPS providers and the message is clear: IPv4 is fast running out. So it's especially frustrating when we can't even get v6 addresses yet.
(To pre-empt, no, it's not ssl web hosting in which I require an extra address.)
You get 16 IPv6 addresses by default, without even having to ask for 'em.
Their legacy virtual machines, dedicated hosts, and the new cloud stuff all comes with native IPv6.
Good company, smart people. (Disclaimer I used to work for them.)
IPv6 has a bunch of possible security issues, and breaks in all sorts of unknown ways.
It's not impossible to do, but it's a lot harder then just installing some software. As an example, libvirt comes with all sorts of network filters that will prevent IPv4 guests from doing all sorts of bad things (IP spoofing, ARP poisoning, etc). In contrast, it comes with exactly zero similar filters for IPv6.
You need to explicitly ask for this from providers. While you might not get it right away, having the demand there is the only thing that's going to encourage companies to spend time on it.
I say unfortunately because IPv4's limitations are going to hurt more and more as the Internet gets more hosts online.
I think the phrase exists in a quantum state of dumb and not-dumb.
PRs welcome :)
I have and know people who use DO but it's mostly for early stage development prior to moving to a dedicated/customers infrastructure.
I didn't find Windows complex -- One nice thing about the MS world is the homogeneity of the stack, which makes a lot of things easier if it works for you. If you want to run something else, you probably don't want to run it on Windows anyway.
Yeah, one of things I miss about the MS stack. The whole freedom is slavery thing. If the MS stack has what you need it usually has a 'this is how to do X' and it works well together.
At least, you couldn't do those things a few years ago.
See https://developers.google.com/appengine/docs/python/tools/li... for a complete list.
But,yea if you need a library that's not on that list and uses native extensions, then you're kind of screwed.
1) Price
2) Scalability/API (As in AWS style of instant spinup/spindown)
3) User experience for the dashboard and creation process
4) Configurability of your instances
DO is currently fighting on 1 and 3 with a dash of two, which I think is very smart. They're not fighting with lowendbox providers for the lowest possible price, but they're competing with linode and AWS for relatively low cost, but with good stability and ease of use. You can't spin up/down quite as easily as amazon, but they're not targeting that exact use case anyway.
I've moved a few things over to DO from linode and will continue to, because although performance is a bit less, the price points are far cheaper. I think there are a lot of people who use Linode/AWS for boxes that just hum along doing some simple server task(s) that would gladly move to DO for 1/4 of the price, as long as the stability/customer service is there.
I read a rumor on a previous post that they were working on a way to customize an instances storage & RAM but at the current low price points, which I think would be huge.
I found the opposite to be true, spinning up instances on DO is quicker and easier, using the web interface or the API. Or are you referring to auto-scaling and such?
I think RAX and Amazon kick ass at 2 and 4, and DO is working on 1 and 3.
curl https://api.digitalocean.com/droplets/new?client_id=$DO_CLIE...
I didn't need to look that up. It's a pretty standard REST API, with a bit of overuse of GET to make it even simpler. AWS's API is a lot harder to use.
If you don't need the advanced functionalities that are only available on AWS infrastructure, go with DO. DO beats the price performance compared to Linode while the API, documentation, and support are comparable.
However, with that said, my complaint is just regarding the article and not with Digital Ocean themselves. I think they're pretty kick ass...
Under the 'competitive advantage' model, it is somewhat unusual that a diversified company like Amazon could be the most efficient vendor at providing a 'commodified', targeted offering like cloud hosting.
This also provides a classic "moat" -- the more sophisticated AWS' offerings become, the more it would cost potential competitors to even attempt to compete with them.
"Amazon, Alibaba and Hetzner each get more NEW servers online in 6 months, then what DigitalOcean created in its entire lifetime".
It's all about the wording.. Not to downplay DigitalOceans achievement, though. But people should look at those statistics in another way...