Goodbye Google Cloud, Hello Digital Ocean
lugassy.net
lugassy.net
But the OP is talking about running dozens of servers, not hundreds or thousands. My own modest apps have held up to the traffic from being #1 on reddit and not fallen over, never been null routed or nerfed by DO, and served hundreds of millions of requests and hundreds of TBs of data. I have nothing but good things to say about DO.
I have no experience running apps at the "millions of requests a second" scale, and I'm willing to accept DO might be a bad choice at that level. But what percentage of apps will ever reach that scale?
There is a high concentration of serious professionals on HN running world class operations. There is also a large amount of startup and single man operations who would hate to write a $100 check.
The thing is, one can only be on one side. Then the other side has nothing valuable to say to you, all their advice is misplaced.
So, to answer your question. It's stupid to ask what apps will reach a large scale. You're either at large scale already or you are not. Obviously, the article is only intended for small scale operations (actually, the entire article is just a generic ad, digitalocean has a nice affiliate program).
GCP shines at a very large scale. For us, what makes the difference, is the spending predictability and massive discounts coming from running preemptible servers or simply committing to certain usage.
GCP is not as service rich as AWS, but most of the offered services are capable of withstanding incredible loads. In our case, while running on tens of thousands of mostly preemptible servers[0], we couldn't pull this off anywhere else (trust me, we've tried).
It's great to know which service works for your loads and make the best of it.
For small projects, especially when I am paying the bills, it is difficult to beat OVH and DigitalOcean on costs.
That said, GCP is still my favorite provider, largely because using GCP makes me a little nostalgic for the time I worked as a contractor at Google - really enjoyed their infrastructure.
There's time you need more flexibility. I'm serving 1.2MM requests per second from 3 GCP regions, managing instances and GKE clusters with terraform, and I cannot see how could I possibly set that up in a resilient fashion with DigitalOcean.
I think DO is perfect for certain scale apps. You usually care about UI things mostly when you spin up couple servers; but when you operate hundreds of machines you need automation.
GCP has it's quirks, e.g. * 130k connections/core limit due to conntrack, * lower networking throughput compared to AWS (16Gbps on GCP vs 25Gbps on AWS), * no support for enhanced networking (haven't tested recent Andromeda 2.1 yet, though) * no way to attach more than 8 local SSDs (arguably a good thing) * etc
So does AWS, so does DO and you have to pick what's best for your project. One thing I like in general here is competition that makes all of those services better.
EDIT: Fix conntrack typo
I don't disbelieve you, I'm just wondering what type of site that is since English Wikipedia is several orders of magnitude lower than that.
I think these days GCP and AWS are more or less on par. One thing I learned the hard way: invest time into calculating your expected cloud spend. Your use case, partners you integrate with, your audience. Those impact significantly on cloud pricing.
Ended up serving most of the traffic from n1-standard-16 or lower.
Isn't the 130K limit only for core count < 8?
From: https://cloud.google.com/vpc/docs/firewalls
130000 per instance for instances with shared-core machine types
130000 per CPU for instances with 1 to 8 CPUs
130000 *8 (1040000) per instance for instances with > 8 CPUs
Even if you're only operating at 1x $5/month droplet (their lowest instance type), you'll still get top grade email support included with your plan.
Getting a response in a few hours is almost guaranteed and often times they'll even help you debug problems that go beyond the scope of their infrastructure.
Compare that to something like AWS which charge you an extra $29/month minimum (or 3% of your AWS bill) to speak to an entry level support person through email.
If Google’s own premium speedy network, anycast IP, global load balancing and security features bring no benefit to your web operations, then your math might be right.
If you need your LB to handle 10s of millions of HTTP/S requests per minute, then what other choices do you have other than Google?
F/W default “Everything within a project should allowed”? Sorry, but no way.
Not familiar with other DO capabilities, but cloud platform is not just about a VPS available throu REST API, it is way far beyond.
The same options every business that needed them has done for decades? Renting dedicated servers én masse, colocating, or running own datacenters? And doing this in multiple countries?
What I'd suggest for a start is renting dedicated servers or colocating in a few places (EU-Central, US east, US west), and running on that.
Dedicated servers, even with having to maintain them, are always cheaper than GCP. Even more so is the network traffic.
I've started with dedicated servers with a few European hosters myself where I'd be able to get servers and traffic for 40€ a month that would have cost me upwards of 14'000€ a month with GCP, AWS or Azure (180TB+ traffic per month, I don't have the exact numbers again, but I used each service's cost calculator for this). Sure, if I want high quality networking, I'd have to pay another 360€ per month extra with these hosters, but that's still cheap enough that it means I can basically run a 180TB/month serving service on a hobbyists budget.
Also, from my experience, when you reach a certain level of operations, you discuss with Google sales team and you get a significant discount for bandwidth.
It's also higher quality of service IMO. When AWS gets their daily DDoS traffic, they manage it and we never notice nor care. With smaller colo providers, they are much cheaper, but if your neighbor is getting DDoS'd, you notice it. I assume Azure and GCP are similar.
On Twitter, the OVH CEO claimed that they've previously had servers targeted with 1.1Tbps DDoS, and still managed to keep that user's site working.
Hetzner and OVH have much better peering than online (ofc still worse than GCP), especially noticeable when users are in Asia Pacific and you have only servers in the west.
Online is significantly cheaper, but well, the peering issue.
Generally I prefer Hetzner for compute-heavy workloads (especially their ML server offer with an i7-6700 and GTX1080 for 99€ is awesome) or networking, for most of the rest, I'd just go with Online.
Important for me is that I can just rent a handful servers, try out building something, fail, and try again, always knowing that it won't cost the world, and allowing me to experiment. If it fails, I can just try something else, if it works, I can just scale up. Doesn't have to be perfect, just has to be enough to let me fail early and often.
DO is slowly transforming into a "light" cloud service. They have load balancers, a cloud firewall that's partially VPC like, object storage, block storage, etc.
It is still far from being AWS, but they seem to be headed, at least partially...that way.
I think there's plenty of room for a budget cloud provider that doesn't kill you with network egress charges.
Yeah no. GCP is a direct competitor to AWS, so of course Google needs (at least partly) to match Amazon's product lineup. The argument of this post boils down to "GCP should be like DO", i.e. less services and a slicker UI, but Google won't win over AWS customers with that.
Also to offer my own anecdotal experiences using DO. I find it highly inflexible when I need to do anything more granular. Sure it's easy for deploying a standard build, but anything that falls slightly outside of DO's remit quickly becomes more trouble than it's worth. Where as AWS (and similar) might intimidate new comers with it's scale and complexity but that flexibility can also be a godsend.
[0]: https://www.digitalocean.com/community/questions/extra-bandw...
[1]: https://motherboard.vice.com/en_us/article/qkj35w/nra-compla...
As someone pointed out you have Lightsail if you want Linode/DO from AWS. 3.2.1... lets have cliche AWS/GCP/Azure is ridiculously expensive based on some undefined VPS memory/cpu details, hey virtual core!, and let's not forget the usual HE.net / Cogent / Level3 quote of 10Gig for $2000 and work out real bandwidth from there.. :rolleyes:
I'd highly recommend people check out the performance of the High-CPU Droplets for any compute-bound tasks like CI/CD, batch processing or active front-end web servers.
Someone from our team put together a comparison of compute series machines which includes GCE and you can see the price-to-performance ratio here is pretty solid for DigitalOcean: https://imgur.com/a/o16t2
For prototypes and small projects dealing mainly in TCP and if uptime is not really important then DO is great. For us, we stick to AWS / GCP for the bulk of our load including dev/testing etc as we want to have the environment as similar to prod as possible.
It's not private. It's internal and shared with all other customers at the same data center location. I wouldn't be surprise if this misleading marketing resulted in lots of accidentally exposes services.
My own needs are modest, but I have provisioning scripts that create and destroy DO droplets on-the-fly from the command line.
Anything more complex, such as weaving several server under one VPC with a load balancer and routing configuration, I'd recommend checking out terraform. It also supports multiple cloud providers.
az container create --name mycontainer --image microsoft/aci-helloworld --resource-group myResourceGroup --ip-address public --ports 80
On GCP you can run:
gcloud app deploy
And it will deploy your app on App Engine, with auto-scaling built in. The thing is, that is just the start. Pretty soon you will need databases, caching, versions, etc, etc.
(I work for Google Cloud)
I've found even some relatively simple infrastructures (2x web servers, 1x database server + some extra caching services), AWS actually worked out cheaper than Digital Ocean. The problem is you'd have to approach hosting on platforms like AWS slightly differently than you'd approach hosting on the likes of Digital Ocean. But once you start scaling your application, you'll soon find that you'll want to adopt the aforementioned topology anyway regardless of arguments about AWS/Google/etc vs Digital Ocean.
It obviously depends on the application but generally you'd expect that to be the case for web sites (ie not services that act only as an API backend).
> 99% of our requests are dynamic (thus the NodeJS frontends)
I don't know your business so you might be transferring more dynamic data rather than static but your argument here doesn't help explain your business as you're mixing metaphors. eg most of our requests are dynamic as well but JSON APIs will generally return smaller chunks of data compared to an image. So in real bandwidth terms on our application, more data is transferred for static content rather than dynamic. None of the static content touches our web servers though. In fact a fair amount of the dynamic content doesn't either since a lot of that isn't user specific so is served from caching services.
If you don't mind me asking, how much bandwidth you transfer each month? Even taking the above into account, our bandwidth costs (which is nothing compared to when we self-hosted) gets dwarfed by our computing costs (VMs and DB).
By the way, how are you finding node.js for your workload? I find it a weird choice if 99% of your requests are dynamic based on bottlenecks I've experienced when deploying test systems on it. But I ask about your experiences because I've not used node.js for anything at scale (ie just a few thousand requests a second).
I’ve had months where I’ve transferred above 180TB of data, at an overall cost of below 40€ in that month, with dedicated servers at a European hoster.