Hacker News Reader

Comment by jomamaxx | Hacker News Reader

Listen folks - it's still a fantasy. Latency is still 5s on many requests = no deal.

This has been going on for years.

Would someone form AMZ please stand up?

I suggest that the latency is 5s for the 'first call', then, subsequent calls are fast - and if you don't use your lambda for a minute or so, it goes back to having latency.

I suggest maybe this has something to do with loading it into memory when it's used?

There must be a way to keep the lambdas 'hot' or else it will never really get used for anything but background tasks.

View on HN

stands up https://www.youtube.com/watch?v=eJO5HU_7_1w

You're describing container re-use which is documented in the page here: http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduct...

You could have cloud watch run the function every minute? Again, that won't guarantee that your function will be "hot". But it's a quick hack to potentially reduce the frequency of your slow initial start.

That said customer feedback is super helpful in making great services so I'll get this feedback to the lambda team (or they may see it here).

If you ever need any help my email is in my profile and I'm happy to work with you.

Thanks for your response.

I checked out the doc and didn't see any reference to latency or how to mitigate it, perhaps I'm missing something?

Listen - I'm a massive fan of AWS. We use some core services and are very happy.

I know you listen to customers - but I suggest that Lambda was probably designed for specific kind of services (maybe IT, admin, batch processing etc.) - I'm going to fathom that maybe 'people running web-sites' were top of mind? Again, purely speculation on my part.

But if you want people to seriously switch over - and they will - in droves - if you get it right, is to basically enable fairly responsive services that scale with a minimum of fuss, latency concerns included.

So - if I'm running a web-portal - and I have a customer pull some data to build a chart from somewhere - and I have a Lambda specifically geared for that microservice - then it needs to be quick.

There really should be no need to use other services to have to hack around the latency issue.

Surely - you can parameterize the feature if you want, and require users to set thresholds for how long Lambda's 'stay hot' (and forgive my crude metaphors), or how much throughput etc. - and you could even bill them based upon those parameters. As long as it's fairly concise and understandable.

Perhaps DynamoDB is a good example. We've used it before - and what we love about it is that as long as we design our data correctly we never have to worry. Bursts of traffic, etc. are generally handled well, it scales largely without concern. One poor limitation of that service is the requirement to indicate throughput thresholds, which is too bad, but beyond that, it's ideal.

Amazon Lambdas are basically the 'last piece' of the golden dream of cloud computing - the ability to develop a truly serverless, microserver-based infrastructure, and never have to worry about platforms again. As much as we love EC2s, we would drop them instantly and go to Lambdas if they worked well. (And it's been at least 18 months since we last experimented, BTW). There are tons of SMEs that would do the same - I don't think anyone who isn't directly in the business of tech, and is providing some other kind of consumer or business service, wants the pain of managing EC2s or servers. 'Elastic Beanstalk' etc. is a bandaid - it's not ideal.

Anyhow - Lambdas are very close to being a super great thing, more than that - transformational - if they could just cross that 'real world' usability threshold.

Again - if something has changed WRT latency, and there is something in those docs I'm missing, please let me know.

Not having PCI compliance in Lambdas are a huge drawback that keeps us on EC2s despite having a large amount of our webservice now on Lambda. API Gateway latency would be a big problem if we were doing a high perf webapp. Fix those 2 problems and we get to do serverless all the way.

You break the entire model of an oversubscribed multi-tenant container system if you want every lambda hot all the time.

If that's what you need...you need a server.

"If that's what you need...you need a server."

We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again.

"You break the entire model of an oversubscribed multi-tenant container system" - we don't care that much about how it's implemented, or what underlying model is used, as long as it is secure, robust, and reliable.

> We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again.

I'm going to argue that's currently unrealistic, but if you want someone to build something, tell them its impossible.

Clearly its not easy to build a system that'll run code on demand as fast as bare metal; accept the trade offs with the understanding why you're making them.

What I'm asking for is definitely within grasp of Amazon's team :) they just need to get there.

Moreover, it's a rather fundamental opportunity for them: Lambda's are a form of 'container' and they are exactly what so many of us want. The deal-breaker for us was the latency issue.

Amazon is a pretty smart company, and it's a big opportunity, I'll bet they eventually get this sorted out.

What you're describing is managed hosting. In order to do it, you either have to oversubscribe like hell, and deal with latency, or pay for wide open swaths of server. Pick your poison.

Keeping a Lambda container in memory is probably not going to take more than 500MB of RAM. Paying a monthly minimum fee per lambda per machine per month to keep it in RAM should be an option. It would still be far cheaper than a real server.

Would you be able to explain why it'd be far cheaper than a real virtual machine? Because the ram is the most expensive part in both cases.

I measured the actual latency over the weekend [0] [1]

Bare ec2: 15ms

Lambda via API Gateway: 276ms

[0]: http://prerender.cloud/lambda-latency [1]: https://news.ycombinator.com/item?id=12678011

Thanks for the data points.

I understand that Lambdas are 'relatively quick' when they are 'hot' - which is my crude metaphor for 'up and running'.

The issue is, if the Lambda has not been in use for some time (seconds, minutes?), then the 'first call' can have quite a bit of latency - often up to 5 seconds - which killed it for us.

There is no way we could reasonably deploy a service wherein customer would have to wait an additional 5 seconds for a reaction in most use cases.

So close, yet so far.

This was about 18 months ago.

If you have some data that addresses this specific artifact - then that would be great. Possibly things have changed in 18 months.

And I'm not a hater, to us AWS is the best thing since sliced bread.

Actually I was making the point that they are not "relatively quick" even when hot: close to 300ms, almost entirely due to the API Gateway. (I even measured API Gateway on top of bare ec2 and the same penalty applies).

Lambda itself is fairly quick (~75ms), but API Gateway needs some work.

Yeah, I tend to agree, I just didn't want to be a total jerkoff on the thread.

250ms we can live with for many things, but not 5x.

But yes - it should be under 50ms at all times I think for people to start taking it seriously.

The 5 second cold request thing is what GAE had 5 years ago. Eventually they gave up trying to make startup faster and allowed a paid plan to stay always hot. Which was good, but then also moving you much closer to a managed server (vs a pure pay as you use thing)

No way you're getting 15 ms on a TLS connection over the internet. I've set up similar pings with API Gateway and API Gateway -> Lambda, and it seems with the APIGW the TLS connection is the most expensive part. On an established connection I've seen 10 ms overhead which is not great but worth the value in some cases.

If you are serving high traffic then what's the problem?

If your product has only occasional use then you trade cost for latency; spin up an ec2 instance if your request times exceed a few minutes call

You could have certain requests that don't come that often, even on a high traffic site. And if you don't know for how long lambdas stay hot, how are you supposed to time it? Even a call every few minutes may not be enough to keep them hot.

Ah, I see. Perhaps that would be a reason to keep an ec2 instance spun up to host the functionality that must always have latency < warmup+runtime? Surely suboptimal. This makes me consider running a cron job to poke timing critical lambda workflows every few minutes...which to me imply writing my functions in such away as to expect this "heartbeat" and act accordingly...and that just feels plain icky...

Reply on news.ycombinator.com