Hacker News Reader

Comment by cddotdotslash | Hacker News Reader

Back when I was working on everything myself, I deployed everything through AWS Lambda and API Gateway, with all my static assets on S3 and CloudFront. I had exactly zero infrastructure issues over the course of two years and never dealt with security patches, SSH'ing, etc. If I were doing billions of requests, it may not have been the most cost effective, but it helped me scale without worrying about typical devops issues. Updates, testing, rollbacks, etc were also extremely easy.

View on HN

How do you keep things fast?

Lambda functions can have cold starts that introduce latency. How do you manage that?

(From my small amount of experience - please prove me wrong.)

If you're doing a SPA, you can paper over the cold starts a bit, since the app itself will render, and it'll be background requests to load data that are impacted.

That still sucks, so then you can (hopefully) cache some things so that _some_ data begins to stream in. Or you can make it so your very high priority stuff has minimal dependencies - you can get a Lambda cold start in < 1s if your app only uses the standard library.

But still, in my experience, cold starts are a thing. If you have a high-traffic app or use Lambda warming, you decrease the # of people who experience a cold start, but at the end of the day, your p99 is going to be worse than a vanilla VM solution, because _some_ people will get cold starts. For some apps, that's OK - think line of business app where the first few pages can be served from static materials or cached materials, and you trigger the requests in the background.

First, caching. Can be done via API Gateway or Cloudfront.

Second, minimizing the function's code size. Both by making small functions and then optimizing them (there are plugins for the serverless framework to do that)

Third, using a language with a minimized cold start. Some can have cold start latences lower than 1 second [^1]

[1]: https://mikhail.io/serverless/coldstarts/aws/

If latency is a big deal, don’t use lambda. I went from not knowing anything about Docker to having a Fargate (Serverless Docker) NodeJS app running in less than a day using a walkthrough I found.

But then again, I did already know the other fiddly bits of AWS.

I am going to add that Lambda cold starts are pretty overblown unless you really do have an application that is super sensitive to latency (i.e. you are doing stock trading or something else pretty latency crazy). Most applications can use Lambda and pretty much never worry about cold starts. Consistent traffic keeps Lambda's warm and I've seen services at their busiest suffer less than 1% coldstarts.

It will only cold start the first time. You also need to make sure that if your lambda can reuse the existing db connection when receiving multiple multiple requests to save time.

I setup a CloudWatch event rule to ping the function every 10 minutes, which keeps the function warm without costing much at all.

Did you ever have any problems with the function concurrency limit?

It’s a soft limit. You can request a larger limit and get it increased within 30 minutes.

Awesome thanks! Sorry to bug but this is very interesting to me. Did you use a framework like Serverless?

No. I’m of the belief that when you choose a platform, go all in. Given a choice between the Serverless platform and AWS’s Serverless Application Model, I would choose SAM.

I also work for a company that pays for the business support plan, they will help you to an extent with any weirdness in their own platform - but not with third party utilities.

Another advantage with using SAM, is that you can create and configure your lambda/API Gateway from the web console and then export your SAM/CloudFormation template.

All that being said, for APIs, I don’t use either. I would recommend using standard frameworks like C#/WebAPI, JS/Node, Python/Flask/Django instead and using the proxy integration libraries that AWS provides. It lets you develop/debug locally like you are accustom to and it gives you the optionality to move your APIs to Fargate (Serverless Docker) or EC2 instances without any code changes:

C#/Web API

https://aws.amazon.com/blogs/developer/deploy-an-existing-as...

Node:

https://github.com/awslabs/aws-serverless-express

Python/Flask

https://dev.to/apcelent/deploying-flask-on-aws-lambda-4k42

I’ve been told that there are similar frameworks for other languages but those are the three that I use.

If you have any other questions, feel free to email me. My address is in my profile. I’m not a consultant trying to sell anything....

Thanks so much! This is really interesting. I think acloud.guru has a similar set up and they pay basically nothing for their millions of QPS. Going to research this more for my next project.

Reply on news.ycombinator.com