Hacker News Reader

Comment by evrydayhustling | Hacker News Reader

That was interesting. Two questions:

1) Is there a problem with python, or a problem with flask? Isn't this what chalice is for? https://github.com/aws/chalice

2) How are you dealing with cold starts?

I learned stuff from this post, but I would have learned more with some background about the workload etc so I could reason about what generalizes and what doesn't.

View on HN

Not the author, but you can set up cloudwatch to hit your lambdas at defined intervals. I set up my lambdas that are accessed through api gateway with a special header to check. If the header is there with the correct value, it just returns. Most keep-alive checks are in the 10-20ms range, and since charges in increments of 100ms, it's the lowest possible tier for getting charged.

We have done this. The problem is that concurrent requests have to warm a new instance - so if your concurrent workload increases, newcomers face cold starts. Worth noting that we are more worried about user experience from slow returns than price.

Edit: forgot to say thanks for suggestion! Also, here is a related article: https://hackernoon.com/im-afraid-you-re-thinking-about-aws-l...

That's an interesting problem. We don't get many concurrent requests and if we do, well then it's not a huge deal.

How many instances do you want running? You could set up a separate keep alive path that sends another request to the lambda, with a variable on how deep into the keep alive request 'recursion' and break out if you deep enough. Does that make sense? Super weird and just off the top of my head.

edit: this isn't a good solution either because if you have a lambda kicking off 4 other lambdas because you want 5 running, and someone makes a request well then you still haven't warmed up that 6th and your 5 lambdas are running the keep warm code...

If I understand your suggestion right, it's to heartbeat concurrently to force more warm instances. We have played with that, but spikes are spikes - the most interesting ones defy expectations. As with many apps, the conditions that make us spike make performance more important, not less.

Just found the same author as OP with a clever solution here: https://read.acloud.guru/cold-starting-lambdas-2c663055589e

Having the app pre warm instances on a per-user basis is super cool -- for user-driven workloads like web servers. To make matters worse, we are serving an API that takes hits from third party streams -- so our concurrency is based on their client behavior, not something we can easily link to a session scope, like users. Tricky!

Yes. That's what I mean. The per user basis does sound interesting.

Sometimes though, you can't force a square peg in a round hole. I dislike server maintenance but docker is a decent alternative to lambdas if you can absorb the extra cost.

Agreed, I think that's the state of the art: if variable concurrency is important, manage your own spare capacity. But I expect AWS and other providers will some day let us pay for reserved capacity without managing it, and I can't wait.

Yeah it seems a like a lot of resources are wasted on useless pings.

It does. But if your endpoint is so inactive that it sits idle most of the time, having it on a server/ec2 instance means you are paying 24/7 for it to sit there not doing anything. You could argue that it's not that much different to pay to keep the lambda warm vs paying for a server to be idle half the time.

Reply on news.ycombinator.com