Comment by munns | Hacker News Reader

Comment by munns | Hacker News Reader

Hey all, I lead developer advocacy for serverless at AWS and was part of this product launch since we started thinking about it(quite some time ago I should say). I'm running around re:Invent this week, but will try and pop in and answer any questions I can.

Provisioned Concurrency (PC) is an interesting feature for us as we've gotten so much feedback over the years about the pain point of the service over head leading to your code execution (the cold start). With PC we basically end up removing most of that service overhead by pre-spinning up execution environments.

This feature is really for folks with interactive, super latency sensitive workloads. This will bring any overhead from our side down to sub 100ms. Realistically not every workload needs this, so don't feel like you need this to have well performing functions. There are still a lot of thing you need to do in your code as well as knobs like memory which impact function perf.

- Chris Munns - https://twitter.com/chrismunns

The cold start from using lambda has a number of causes

1. the time to initialize the VM

2. the time to create an ENI if you are connecting to a VPC[1](until the NAT alternative rolls out globally)

3. the time to initialize your language runtime (Java seems to be the worse, scripting languages the best)

4. any program initialization done outside of your handler that runs once per cold start of your lambda runtime.

A fully “warm” instance avoids all four when run.

Is my understanding correct that a “provisioned” runtime that isn’t “warm” will only avoid the first two?

What state is a “provisioned” instance in?

[1] I refuse to use the colloquial but incorrect statement that the lambda is “running inside your VPC”.

Fwiw new networking for VPC is completely rolled out for all public regions now. (#2) (and thank you, its called "Attached to a VPC" and not in :) )

This covers you straight through 4.

Now it's possible that your execution environment could be sitting for sometime waiting for any action and so pre-handler DB connections and things like that might need to be tweaked in this model.

Thanks, - munns

So I had to convert three lambda APIs using proxy integration to Fargate mostly because of the 6MB request/response limit but the cold starts caused us to make a rule that we weren’t going to convert our EC2 hosted APIs to lambda. We were going to host them on Fargate.

But since the APIs that I moved over to Fargate are now automatically being deployed to both lambda and Fargate with separate URLs, we can A/B test both and see if we will move to lambda in cases where the request/response limit isn’t a problem.

Btw, I didn’t think using a NAT instead of an ENI had rolled out completely. I tried to delete a stack recently and it still took awhile to “cleanup” resources. I thought that was caused when it was deleting the ENI. I’ll be on the look out for it next time I need to delete a stack.

Well, now we have RDS proxy that was announced after my original post, so that should help with DB connections....

I'm wondering how feasible it'd be to vary PC through the day.

The pricing examples include using PC on a limited duty cycle, and billing is defined to start from the moment it's enabled (rather than from when it's ready), so it'd be reasonable to expect there's some level of certainty that the concurrency level is ready within a defined timeframe. What might that timeframe be, and to what level of certainty?

PC supports auto scaling.

https://docs.aws.amazon.com/autoscaling/application/userguid...

Sure, but there are no actionable metrics re. response time.

The closest is a suggestion from prior documentation[1] that Lambda "can scale by an additional 500 instances each minute" for a given function, but that's phrased like a promotional claim, not a commitment or even an objective, and it's even unclear whether that's a floor, a ceiling, or some average measure. I wouldn't doubt that PC lives on the same control plane as Lambda's regular scaling, but assuming identical behaviour is unwise unless documented.

Guess I can suck-it-and-see.

[1] https://docs.aws.amazon.com/lambda/latest/dg/scaling.html

We've just published a second post on this launch: https://aws.amazon.com/blogs/compute/new-for-aws-lambda-pred...

Reply on news.ycombinator.com