Hacker News Reader

Comment by jifucboc | Hacker News Reader

It's been a while since I read such a long and detailed blog post (a 6 month effort according to OP).

> With the default limits, Lambda scales to 3 000 containers in a couple of seconds. That means that with default limits, we get 30 TB of memory and 18 000 vCPUs in a couple of seconds.

> With a legitimate workload and increased limits, as we just saw, we are now living in a world where we can instantly get 150 TB of RAM and 90 000 vCPUs for our apps

Going from 0 to 150TB instantly is powerful, but I don't think there is anybody that needs that.

View on HN

It's nowhere near these volumes, but at work we recently experienced traffic spikes like 35-fold in the span of 30 seconds for a few minutes, then faded regularly for a few hours. Our scaling was too slow to be of any use for such pattern. There are legitimate needs for things that scale as fast as possible.

E-commerce flash sales, DDoS/spam prevention (where you're looking at the content, not just blocking traffic), data analytics. Not only for the e-commerce site but downstreams systems like payment processors.

There's also downstream infrastructure, too. If your website scales, you now have a bunch more log and metric data--what do you do with that?

A lot of times, data analytics "can be slow" but a human is sitting around waiting for processes to complete. A boost in speed usually means a boost in productivity.

There's also IoT. I think ~2012 GM was dumping terabytes of engine sensor data per airplane flight. Being able to process that quickly could mean the difference between being able to adjust engine fuel efficiency between flights, for instance.

I think telecom/voice/audio processing have pretty tight latencies. Even "second" latencies are probably too slow for those types of systems.

In addition, reducing scaling time potentially means reducing scaling cost since you're paying for compute you might not necessarily be using (depends on the billing model).

We have a compute job scheduling system that often sees swings like that (bigger, actually!), where it goes from idle to having a ton of newly queued jobs ready to run. But we use EC2 spot instances because they're much cheaper and the resource limits are much higher. It wouldn't matter if Fargate scaled out arbitrarily high, arbitrarily fast -- that's not really the problem with Fargate for our type of usage.

We do orchestrate the EC2 spot instances ourselves, because ASGs and Spot Fleets are way too slow. Spot Fleet especially is like calling your grandma to have her click around in the AWS Console every time you want to scale. AWS is way too "polite" to their own infrastructure but when you do it yourself, you can slam right up against your rate limits.

I can think of an example; the gambling industry. When a large event (such as a big football match, the Grand National, etc) finishes, all the bets will need settling. It's difficult to do big swathes at once, since different accounts might qualify (or not) for promotions or similar.

Any other large-scale individual action driven by a particular event happening could also fall under this description; concert tickets going on sale for example.

Of course many workloads might not need to go from zero to 3,000 containers that quickly - but it's useful to know that the underlying infrastructure can do this so that you know it ought to handle far smaller demand spikes without much concern.

Reply on news.ycombinator.com