Hacker News Reader

Comment by moqizhengz | Hacker News Reader

Awesome work, but I am genuinely curios about the use cases where the 200ms init time being a problem?

Serverless implementations where lambdas are being used for APIs so +200ms on a single call is not good.

It's not 200ms per call. It's 200ms cold start. Once a lambda is started, you don't pay that cost again. In most cases, you pay this a few times after a deployment or when traffic is increasing and not again.

> you pay this a few times after a deployment or when traffic is increasing and not again.

This is now how lambda works. A lambda instance sticks around for 10s of minutes at most. So you'll have cold starts every day no matter how often you deploy.

If you have very little traffic you might actually have cold starts often as the lifetime of a lambda is determined based on its use.

Unfortunately AWS has always been rather unspecific about this behavior but here's a link to a pluralsight blog https://www.pluralsight.com/resources/blog/cloud/how-long-do...

Maybe its a problem when you’re paying 200ms per small request and you have millions of them ?

Some companies who actually depend on actions like this should consider fronting some cash to keep this project going.

If you have cold starts on all of your millions of requests, something is very wrong with your code.

The hypothetical 200ms is for cold starts. That means when the docker container that runs behind the lambda first starts up.

When a lambda is invocated, that docker container isn’t immediately destroyed. It’s kept around for reuse.

So there isn’t a realistic scenario where you would get a million cold starts in a month.

I have a lambda that runs inference on a model for a feature that is seldomly used.

Lambda is far more economic given the memory requirements, but we recently moved to Rust+Candle to shave off ~300ms on cold starts as the lag could be really jarring.

Reply on news.ycombinator.com