Hacker News Reader

Comment by rad_gruchalski | Hacker News Reader

Oh dear. “$87/hour, which is $62.6K/month” for only 250k concurrent connections?

Around €1500 will get you a second hand Dell R720 with 190+ GB RAM and 48 cores.

It would be cheaper to handle such scaling with a vm instead of functions. Any kernel with a bit of tweaking will easily handle that number of connections. One just needs RAM and sufficient CPU. 16GB RAM and 8 vCPUs on aws will be enough for 500k.

Sleeping would be very uncomfortable if my service using such approach started being popular and was running at increasing capacity for a week because my team needs time to move back to vms.

View on HN

> It would be cheaper to handle such scaling with a vm instead of functions.

Ahmet makes that exact point:

> In the long term, as your load becomes more predictable, it makes more sense to move to VM-based compute (such as GCE or GKE) as several mid-size virtual machines can handle the same load, potentially 50x cheaper.

The shiny spot for something like Cloud Run -- or Knative -- is varying workloads. I described this as "workloads with unpredictable, latency-insensitive demand" in my book on the subject[0].

[0] https://livebook.manning.com/book/knative-in-action/chapter-...

What I never get with these is: one has to really go out of their way to run the same code on a vm and serverless. So why even bother building the thing twice?

Do people really consider putting a server app in a function?

Cloud Run uses docker containers, so the code can very easily be the same.

I'd put a server app up behind a cloud run endpoint, sure, as long as it can fit in a few GBs and start in a few 100 ms. The scale-to-0 aspect is nice, and the scale up aspect is nice too.

For me the selling point of cloud run is that is easy to develop and debug: make it run locally, test it, then ship it in a container.

For GCP cloud functions, AWS lambdas, Azure cloud functions you must fight somehow against the environment (that's the reason frameworks like Zappa exist).

If you want/need to use compute instances, you can use KNative in your K8s cluster and enjoy a "similar" scaling experience.

I’ve seen ory hydra being fir in a lambda function and as awkward as it looks to be, it works reasonably well.

Thanks, good to know it’s a common technique.

This is the beauty of cloud run. There is literally no difference in how you build your application. I can run my codebase locally using docker with no changes and no dependencies on the type of external libraries you'd need for AWS lambda or cloud functions.

> So why even bother building the thing twice?

Time elapses. What makes sense today may not tomorrow. It's fine to adapt designs when the economics point that way. Also fine to leave things as they are when the economics point that way.

Yep we’re running our API at work in a lambda behind API gateway.

It’s “interesting”

Sounds fun! Could you elaborate on the “interesting” part a bit more?

On one hand it’s very easy to run the code locally or in lambda because we use Go along with a library from AWS that translates the gateway proxy request event to a normal http request. So the request handling is all the same in code.

To prevent 404s hitting your function you have to define your routes in both your code and in api gateway. This starts to marry your code to your deployment.

Observability is now harder and you need yet another thing. We’re using X-ray with datadog.

To “safely” talk to Postgres you should use an rds proxy. It’s still unclear how much is different from pgbouncer or similar but in any case it’s pretty easy to knock over your db with runaway lambda invocations.

I could go on about the mistake we made of using SAM and how it ends up not supporting everything you need but I guess my main point is it’s easy to get sucked in by the idea of it being easy but you are trading that for service / operational complexity with a healthy dose of platform lock-in.

This is a pretty good example of how Cloud Run is better, as those connections to the database would be divided by 250 (lambda supports no concurrency). Lambda supports docker containers now but Cloud Run was also ahead of the curve there.

Yep. Running adhoc containers on demand is clearly the right pattern. Still plenty of sharp edges but gives you a very clear path to lift your system admin / structure skills in to "serverless".

I need to spend some more time playing with Cloud Run because I wasn't aware of the concurrency bit you are mentioning.

Oh god, when's the funeral for your bank account?

Yes it's called a fat lambda

I think the specific issue here is the 250 connection limit, which seems to be really under utilizing the instance in something as basic as chat.

This is useful for services that have short surges but very little traffic around the month. If you only need a surge for 1 hour per month, then it should cost around $100/month.

> Any kernel with a bit of tweaking will easily handle that number of connections.

I don't think tweaking a kernel and "easily" fit together. Unless you are kernel developer.

>> Any kernel with a bit of tweaking will easily handle that number of connections.

> I don't think tweaking a kernel and "easily" fit together. Unless you are kernel developer.

I think he means twiddling the kernel knobs in /proc to raise the limit of file descriptors. Last I checked, most distros default to 100k maxmimum file descriptors, with a different (10k) max per process.

Changing the max by 'tweaking' involved echoing a new number to a file in /proc.

It's called a C1M problem. From 2013: https://www.metabrew.com/article/a-million-user-comet-applic.... It's pretty easy. Been working on a product doing this for 500k connections per machine (MQTT) back in 2014. EC2, m3.xlarge instances back then.

Serverless is for uses cases with unpredictable load.

Bare metal is cheaper, but you're paying 100% of the time for your peak traffic. If you only have a couple of peaks with 250k concurrent connections, with serverless you'd pay much less than €1500 but still be able to satisfy that demand.

Taking steps a bit further, it would be interesting to know what variation saves what money. Eg if your load is 80% of the time between 80-100k connections but the 20% at 250k, you save $500 a month using serverless or vice versa.

Is it as simple as $60k/number of connections? If so the cross over point would be just 6250 connections as your base load. IMO, that $1500 server starts to look very appealing very quickly but I imagine it isn't as simple as that.

Adding to this, how many watts is this consuming?

epoll allows high concurrent client count, but how many requests per second and more importantly how many requests per watt!?

Also what is the response payload size?

I can do 1000 HTTP req/res (4K) / watt on my new atom 8-core server with my own HTTP app server passively cooled and tiny mini-itx size!

where can you find servers like that for that price?

servershop24.de; not affiliated, just a happy client.

And as hosting Hetzner.de

Also happy client im easily handling hundred thousand connections for less than 80€/mo

Love to read more about these stories rather than the same crazy cloud costs that get normalized all the time. Just by looking at Hetzner and others, I can see this kind of thing would be extremely doable but concrete stories of those doing it would be something I think a lot of people would enjoy reading about trade offs etc.

Reply on news.ycombinator.com