Autoscaling, Welcome to Google Compute Engine | Hacker News Reader

Autoscaling, Welcome to Google Compute Engine

googlecloudplatform.blogspot.com

68 pointsAndrewDucker31 comments

This has been a long time coming. The autoscaler has been an interesting, and possible the most interesting, part of AppEngine and IMHO has recently reached the point (in the last year-and-a-half) where I'd say I'd want to rely on it for production.

It wasn't always this way -- at one point we were having so many issues with AppEngine's autoscaling that we ended up in a meeting across from Ben Traynor and heard some of their engineers discussing post-mortems of scaling failures in gruesome detail. They've come a long way since then and AppEngine has really found its way from something of an internal science experiment to what I think is the best competitor for AWS' offerings.

FWIW, the "backend" or "basic" scaling module that was available for both AppEngine and GCE was so poor and unreliable that it made it nearly impossible to scale anything in a way other than manual. I honestly hope they deprecate this and roll the few features into the autoscaler.

I'm confused - are you saying that scaling was horribly broken for App Engine, not just Compute Engine? Because it seems that it's much simpler to scale App Engine apps and App Engine has been around a long time.

Autoscaling was initially available for AppEngine only and the same (or similar) infrastructure is being made available to Compute Engine. Compute Engine had access to "basic scaling" which was so horribly broken and unreliable that you may as well just set up a static number of instances.

The entire scaling infrastructure of AppEngine was horribly broken and unreliable up until a year-and-half or so ago, but they've put a lot of work into fixing that recently and it's in awesome shape now.

So, I'm actually intensely curios about how this thing works, even though its probably a trade secret or something. How do you prevent oscillations in resource allocations when the traffic is fluctuating? How do you know if e.g. an adversary is just DDoS'ing your application to make it more expensive to run your application? I guess there must be some kind of threshold that you can set? But then this would cut off the website for the real customers...

> an adversary is just DDoS'ing your application to make it more expensive to run your application?

It isn't the compute engines job to determine if workload is malicious. That is a separate but real problem.

PM for GCE Autoscaling here. On protection from malicious traffic spikes - you can set a max-instances threshold to prevent over-spend which you can change at any time.

Ultimately though, from an instance's perspective, there's not a lot of difference between a malicious DDoS and a genuine traffic spike. You can mitigate this (a) by setting low max-instances thresholds when you aren't expecting high traffic, and (b) upstream filtering or QOS management to filter our DDoS traffic from legitimate traffic.

Do you have models for traffic pattern differences between spikes and DDoS attacks? It seems that a DDoS attack would have different patterns (quicker spike, no buildup, etc.). Just speculation here.

Autoscaler will respond differently to different traffic patterns, but won't go as far as identifying a particular traffic pattern as malicious and refusing to grow resources to accommodate it (most architectures filter malicious requests upstream of the instance itself, eg. at the load balancer or via a proxy).

Even if you could identify malicious usage, it's still a matter of policy on how you want to respond to it - given that if it can reach your VM it will likely be affecting legitimate traffic too.

Would it be possible in the future to set the thresholds based on time?

Say I expect high traffic only during mornings, and not want to handle it during the evening if it occurs. Any chance this is in the making?

> its probably a trade secret or something

You can do auto-scaling without Amazon's secret sauce (or Google's), it just relies on triggers to spin up new boxes. I'm not very familiar with the Google setup, but you could use the Amazon API to pull metrics about boxes, then send a request to add another box to the auto-scaling group manually if you wanted.

> How do you prevent oscillations in resource allocations when the traffic is fluctuating?

Oscillation is kind of the point -- if you need x servers to serve y hits per hour, you'll have x servers when you need it. But when you're down to y/2 hits per hour, you don't need to be paying for x servers anymore. It just depends on load -- for example, you can set the group to add a box when average CPU rises above 70 percent, and then set it to remove a box when average CPU falls below 30 percent.

> How do you know if e.g. an adversary is just DDoS'ing your application to make it more expensive to run your application? I guess there must be some kind of threshold that you can set? But then this would cut off the website for the real customers...

This isn't a problem for auto-scaling, it's a problem for web stacks – the whole point of the "auto" part of it is that increase traffic means increased resources. Going counter to that gets complicated quickly. I'm sure there are folks who know more about DDoS mitigation than I do, but putting protection in front of your app with something like Akamai should help, as can keeping an eye out for bizarre spikes from certain IP ranges.

It's worth noting that the GCE Autoscaler is configured quite differently to the AWS one. For the latter you define explicit scale conditions and triggers. On the former you define a utilisation metric and a target.

Resource scaling is not that tough a problem computationally. The tough aspect of it is the practice of defining the driver of scale needs. Past that it is just calculating based on triggers a continuous or discrete scale. I have done it for FTEs in a huge mining operation and it worked like a charm (of course the requirement for scaling was not as real-time as in this case.) Use cases I had to also cater for in that case were:

- reallocation from pools before scaling out, which is not very complex either, but it adds a bit of complexity, especially if you optimise for cost along the way.

- linked scaling requirements, e.g. a team grows but there is a bottom and top limit to team size and when you get a new team you also need a supervisor

On oscillations, you can weather them out by the way you treat your driver sampling window (static or variable) and a cost function. Normally oscillation is not bad for the resource allocation scenario, it is the intention of it, but for FTEs the cost can be more difficult to manage than for computational resources.

Would love to see autoscaling feature in Heroku. Wonder if it breaks their revenue model somehow to have something like this. May be it's more of a complex problem. As increasing dynos may not help for apps that are CPU bound.

I've wondered for a really long time myself https://news.ycombinator.com/item?id=7104653

This is huge. When people ask me how someone could be a true competitor to AWS, my first is answer is usually "autoscaling". It's the feature that AWS has that no one else does (now did).

Azure has it: http://azure.microsoft.com/en-us/documentation/articles/clou...

And based on the comments, has had it for 1-2 years now.

So does Rackspace, apparantly: http://www.rackspace.com/blog/easily-scale-your-cloud-with-r...

It does, but they don't have nearly the capacity footprint that Amazon has.

Azure has it and is a great platform for .Net workloads, but it's pretty terrible for Unix/Linux workloads.

After last week, that answer for me is now going to have to be "code deploy"

I'm pretty "meh" about CodeDeploy; though services are definitely what set AWS apart from competitors, moving code into production hasn't really been a pain point, especially with the "immutable infrastructure foreverrrr" paradigm they've been pushing and now seem to kind of be ignoring. Auto-scaling group launch configs make it easy, and Elastic Beanstalk works well for certain setups.

What percentage of (and which) AWS features do you think are utilized by the majority users of AWS?

I think most users use very few features, mainly because they haven't embraced a "cloud native" approach. Most people just treat EC2 like a rented datacenter instead of using the whole platform.

We should get a beer. Can't remember your email address...

it's in my profile, but it's jedberg@gmail.com

So the "Google Grid"[1] is finally complete?

[1] http://en.wikipedia.org/wiki/EPIC_2014

Is this the same sort of thing as AWS's Elastic Load Balancer service?

It's like AWS EC2 Auto Scaling. GCP also has Load Balancing which complements the autoscaling feature, which is like AWS ELB.

No, it's not the same as ELB. It's similar to Amazon's auto scaling feature that comes with AWS Auto Scaling Groups. There's a key difference, though. AWS ASG is threshold based: users define actions when specified metrics exceed predefined threshold. In contrast, Google's autoscaling is goal oriented: users do not define action. They simplify specify desired values of specified metrics. Google takes care of scaling up or scaling down behind the scene. IMHO, Google's approach gives GCE more room for future optimization.

Amazon Web Service plays the strategy "catch me if you can" (e.g. new services launched last week). Google is trying to catch up, if they want to beat AWS they would need quite a few differentiators (e.g. like buying Firebase).

The problem with this statement is that Amazon is largely matching firebase functionality. They just announced streaming JSON out of dynamodb, and the new lambda functionality means you can trigger events on changes and push out updates. It wasn't explicitly stated but I viewed this as a firebase competitor.

Reply on news.ycombinator.com