Hacker News Reader

Comment by bsbechtel | Hacker News Reader

Has anyone heard of implementing an architecture where you have normal server backend architecture, while using serverless to handle traffic spikes and any failover? If you properly modularize your backend codebase, it shouldn't be excessively costly to duplicate it across a serverless architecture and in some sort of node instance.

My biggest issue with serverless is the high cost at scale, which could be mitigated with this type of architecture. Are there any other benefits or concerns regarding something like this?

View on HN

Sounds more expensive than scaling up more servers on demand.

What I hear a lot is that the scaling can take too long. I wonder if a hybrid where you use lambda as a temporary buffer would be the most cost effective.

In my experience using AWS, you can spin up new autoscaled instances pretty quickly. I don't even have a custom AMI, I use a generic image and run startup scripts to kick off my server software. Takes < 3 minutes to spin up a new one and start accepting requests.

If that's not fast enough, you can bake an AMI and scale more proactively (e.g. scale when your server is 70% utilized rather than 95%)

Actually we have started doing some initial exploration of such an architecture. But don't have results we can share yet. Maybe in one of the follow-on blog posts :-) I agree that it could make a lot of sense in some cases.

I would be interested in hearing how it goes :) Do I follow the OP, or subscribe somewhere else to get notified when you release it?

why would you do this when regular autoscale could just be used instead?

Autoscale doesn't allow you to scale to zero (as any requests would fail, rather than wait on resources to scale up).

EC2 autoscaling takes a minute or two. This means you need to run with enough headroom to absorb spikes before you're able to spin up more resources. Depending on traffic patterns this can get very expensive (or not a problem at all, with predictable loads).

if you're worried about the "zero" traffic situation I am not sure you also need to worry much about the "massive" traffic situation

I am sure :)

Our customers are concentrated around N. America, with minimal activity at night and weekends. We have a complex product with a wide surface area. Some of our services don't see usage constantly throughout the day.

Our usage is unpredictable and spiky, worsening our ability to cost optimize EC2 autoscaling.

Then there's staging environments...

The smallest ec2 instance is $4/m on-demand (or $8/m for a usable amount of memory)

>Our usage is unpredictable and spiky, worsening our ability to cost optimize EC2 autoscaling.

Smallest EC2 instance will fall over with a sudden traffic spike, and requests will fail for the couple minutes it takes for autoscaling to complete.

Additionally for HA, a machine must be able to die without exposing us to the risk of a traffic spike taking us down.

Actual use case I heard of: Outage reporting system for a utility. You know, the page you go look at when your power goes out. The majority of the time, this page receives no traffic. When there’s an outage, the traffic spikes immensely. When the outage is over, the traffic drops back to near-zero.

An interesting way to work around this would be to have the ability to scale with mixed instance types.

For example, your base application runs with t2.micro. As soon as traffic starts going up, you launch additional medium/large instances. The load balancer would have to understand this also, and do weighted routing (ideally based on app server loads) so the first micro instance doesn't handle the same volume of traffic as the rest.

I've never seen a setup like this before, and frankly, part of me wonders whether the savings would be worth the effort of building it in the first place (vs just running with medium instances, or just scaling to dozens of micros to handle the load, or just using serverless).

All you need is a CDN. Cheap, easy, and decades of reliability. You don't need infrastructure for this.

Every company has parts of their system that has "zero" traffic from time to time.

Reply on news.ycombinator.com