Hacker News Reader

Comment by FBISurveillance | Hacker News Reader

More of a rant.

There's time you need more flexibility. I'm serving 1.2MM requests per second from 3 GCP regions, managing instances and GKE clusters with terraform, and I cannot see how could I possibly set that up in a resilient fashion with DigitalOcean.

I think DO is perfect for certain scale apps. You usually care about UI things mostly when you spin up couple servers; but when you operate hundreds of machines you need automation.

GCP has it's quirks, e.g. * 130k connections/core limit due to conntrack, * lower networking throughput compared to AWS (16Gbps on GCP vs 25Gbps on AWS), * no support for enhanced networking (haven't tested recent Andromeda 2.1 yet, though) * no way to attach more than 8 local SSDs (arguably a good thing) * etc

So does AWS, so does DO and you have to pick what's best for your project. One thing I like in general here is competition that makes all of those services better.

EDIT: Fix conntrack typo

View on HN

>"GCP has it's quirks, e.g. * 130k connections/core limit due to conntrack, * lower networking throughput compared to AWS (16Gbps on GCP vs 25Gbps on AWS),"

Isn't the 130K limit only for core count < 8?

From: https://cloud.google.com/vpc/docs/firewalls

130000 per instance for instances with shared-core machine types

130000 per CPU for instances with 1 to 8 CPUs

130000 *8 (1040000) per instance for instances with > 8 CPUs

What do you do that serves 1.2mm requests / second?

I don't disbelieve you, I'm just wondering what type of site that is since English Wikipedia is several orders of magnitude lower than that.

Ads, one of the most shitshow industries in the world.

Forgot to add a joke: you can judge from my nickname! :-)

Can add to that, I worked for a few months in a small mobile ad network and the throughput of that was insane for the size of the company. Also a shitshow of a company on many levels.

1.2m request per sec of ads looks like adblock isnt working .

Good news to me. Let everyone else keep finding all the free services. Everyone can't use adblock or nothing is free anymore.

So Google built a platform that is excellent for running ad networks. It seems so obvious now.

Ads are a super intensive workload with super low margin. It'd be more correct to say that google built a very efficient platform that can run anything. Well, anything easier than ad, so almost anything.

You do get additional 2-3TB per instance which alone is a couple hundred $$ in bandwidth savings. Will it cover the price of these extra instances? Obviously if you need 8 SSDs DO isn't for you. But the real question is - could GCP be more lean back and still provide the same level of service? I think so.

Anyone who count in a unit less than kilo dollar is a waste of time for AWS/Google. A big business doesn't care about "hundreds of dollars" of savings, it's like cents.

It's interesting that 130k connections/core is limiting to you. Care to share some more details? Are those websocket connections? If yes what do you use as your backend? Something custom?

Those mostly are slow user connections. Most of the direct user connections to our backends performed using slow mobile connectivity so request taking 4 seconds considered normal and that exhausts connection limit.

Ended up serving most of the traffic from n1-standard-16 or lower.

so it seems in general you like GCP more than AWS? What is your opinion?

Personally, not professionally, I like GCP more. Feedback cycle is (used to be) faster and it feels (used to feel) less enterprisey. I also liked the idea of a new player on the market, and wanted to make sure I vote for this diversity with dollars.

I think these days GCP and AWS are more or less on par. One thing I learned the hard way: invest time into calculating your expected cloud spend. Your use case, partners you integrate with, your audience. Those impact significantly on cloud pricing.

"used to" - correct. my thoughts exactly 1 year ago: https://news.ycombinator.com/item?id=12235003

Also, some GCP folks are hanging out here and it's nice to have some insights into platform internals.

Yep, pretty regularly I see "Disclaimer: I work for Google Cloud" on HN, with AWS and Azure I see a lot less, no solid metrics here. I also like I can get staff on a Google Groups for many of their products like StackDriver to share feedback and get a PM to respond eventually.

I'm sure there is some internal rule (contract/NDS) at AWS that you are not allowed to talk in any way outside of Amazon about AWS without supervision ;)

Reply on news.ycombinator.com