Why the serverless revolution has stalled (2020) | Hacker News Reader

Why the serverless revolution has stalled (2020)

infoq.com

57 pointsingve68 comments

I don't really agree with the critiques. E.g. GCP will run a Docker container in a serverless fashion, so you can run any language you like using that.

The big issue, from my point of view, is that the programming model takes all the problems of microservices and adds more. Microservices make even trivial features into a distributed system, bringing all the difficulties of reasoning about code that distributed systems entail. Then add an opaque runtime and almost certainly a number of vendor specific services to handle state and other features the serverless model cannot address directly, and you have a recipe for slow development and frustrating debugging.

I don't see serverless as having failed at all. FAAS is a great model, you can run a functional monolith and easily break pieces off or integrate stuff in different languages without changing patterns. Doing it with cloud services like lambda can be expensive but if you have kube setup it has the best of all worlds.

I don't think it's failed, but I do think the "make everything serverless" hype is overcooked.

Same here! We're fans of Cloud Run at my company, but it'd be great if they made it easier to pull messages from Pub/Sub using Cloud Run. Sometimes you just don't want push.

That’s interesting to hear. What problems did you have with push? Having used this setup before, I found it pretty easy and the only time I found myself reaching for pull (on a service deployed to K8s rather than cloud run) was to achieve greater throughput. Was it the same story for you?

Maybe I'm doing something wrong here, but one issue I've faced when integrating third-party external systems with Pub/Sub and Cloud Run is the lack of an easy way to throttle the data flow when the external system falls behind. It would be really helpful if the Cloud Run service could easier support pull from Pub/Sub instead, allowing it to consume messages at a pace the external system can handle.

I could of course do flow control myself in the Cloud Run service, but it'd be nice if I could just switch to pull.

The pricing model of serverless platform is enough for me to not want to use them. Being charged for every little thing your app does is crazy especially when you see horror stories of runaway bills because a function has a simple bug in it that you didn't catch and overnight it ran up your bill so high you can't pay it and are now praying the company feels bad for you.

I'm scared to use serverless platform unless I don't have to put in my credit card lol.

I feel safe paying the same price every month despite my app growing. Servers are powerful these days and are very simple to maintain and scale using cloud platforms because they do it all for you lol.

I enjoyed Google's GCP for quite a long time until they removed the ability to cap expenses with a budget limit. (It used to be that if you hit your budget limit you can make your site error out). Now everyone on GCP is one DDOS away from a nightmare cloud bill. I'd rather use a traditional server and be able to sleep at night. I moved all my sites and client sites off GCP.

The most annoying part is that Googles infrastructure for cloud computing is so much better than the others if you're willing to work within their ecosystem. Simple deployments, version management, rollbacks, etc... There is nothing quite like it. (Im not saying there aren't competitors, just that Google seems easiest to use)

> Simple deployments, version management, rollbacks, etc... There is nothing quite like it.

We do all of this with Azure

Article is 3.5 years old, best put the year in the title, OP!

I think the definition of serverless is too narrow (AWS lambdas/ Azure functions) and that serverless is really "I want to build apps, not manage infrastructure". That's not the same thing as putting everything onto functions.

I have a monolith we're preparing to move onto a managed container system (prolly AWS AppRunner). I don't want to manage K8 if I can help it, our app doesn't need complex server architecture.

Personally, I think it's just the next layer of abstraction up. Some won't benefit from serverless in the same way some are better off with in-house tin. I know some that need custom chipsets in-house, so can't even buy a stock rack server! However, many many web apps don't really need the control and will probably use facets of serverless over time. But it is still a revolution for old people like me that just don't want to manage servers anymore!

    > (prolly AWS AppRunner).

AWS right now feels like the most lagging in serverless containers.

If you get a chance, try out the DX with Google Cloud Run or GKE AutoPilot. Building and shipping to GCR is fast enough that it feels like a local build-run worflow. Google Cloud Run jobs are also fantastic and you get a pretty hefty free monthly grant (~60h of compute).

Both Azure Container Apps and GCR are true scale to zero whereas AppRunner is not and always maintains a minimum monthly baseline cost (~$5-6).

AWS feels the most behind in terms of its serverless container workload experience (I use AWS every day professionally, but use GCP and Azure for side projects).

Thanks for the tips! I'll admit not having done enough research yet - you've helped me do some shortcutting.

> its roots can be traced all the way back to 2006

Serverless was already a thing in 70s, in the mainframe era. In the 90s the pendulum moved to servers and server farms and nowadays is sliding back.

> The Problems With Serverless: > Limited Programming Languages > Vendor Lock > Performance

None of this is a real problem, the only thing that matters is the cost-effectiveness, and currently (2024) it's stil cheaper to run most of your software on "traditional VMs".

> Is This a New Idea?

> The concept of letting users pay only for the time that code actually runs has

> been around since it was introduced as part of the Zimki PaaS in 2006, and the

> Google App Engine offered a very similar solution at around the same time.

You could argue the way you architect "serverless" services feels very similar to the way we built CGI scripts in the 90s. If you kept those scripts small, everyone of those would have been the equivalent of a serverless function today.

This was, in fact, how web 1.0 worked.

The whole dynamic web was bunch of PHP scripts in your home folder.

Even the early Amazon was a collection of Perl scripts glued together.

Didn't old-school mainframes with discrete transistor CPUs use time-share mechanisms and charge enough for the time that people cared about minimizing it? UNIX goes back 50 years, but IIRC there was a substantial amount of "pre-history" before that too.

What's old is new again.

Serverless a simple idea that copying a snippet to the cloud and auto-scale.

You know we used to place the code in .php files and ftp it and call it a day. There were tons of php hosting (anyone remember Dreamhost?) that can auto-scale in some way and bill by page views.

PHP + MySQL + Plain jQUERY. The stack of champions. Seriously. Robust. Simple. Always just works. Minimal dependencies.

For us, serverless is running Django monolith on AWS Lambda in a container. and we LOVE it.

I think the serverless revolution is here just not in the way people originally dreamed.

How are the cold starts? I tried a similar setup with GCP cloud functions and I was running into (I shit you not) 5-7 second cold start times.

There will likely be very few cold starts when the whole monolith is running in it

At that point you’re just paying an exorbitant rate for… the same thing you would get from hosting it on EC2

Or if you can take a bit of up-front setup cost, running it on GKE with Autopilot means you've got many of the benefits of lambda on a K8s cluster (e.g. immutable container-driven development, no need to manage long-running VMs) and only pay for used resources (not idle machines). The main caveat is no scale-to-zero, but as mentioned previously, that's unlikely to happen.

IMXP, autopilot is multiple times the price of a VM on a non-GCP platform.

How come? I see Autopilot is $0.10 per cluster hour, or ~$75 a month, add the standard GKE cluster management overhead of $150, that's $225 a month for a cluster which rounds to zero given any reasonable load. Then in my experience GCP VMs have always been price competitive with basically any other cloud provider.

The issue we had was getting the bin-packing right on our cluster, and we often ended up with 20% unallocated capacity per VM. Autopilot basically solves that (and probably pays for itself with only a few VMs) as you only pay for the K8s defined resources, and therefore only have 1 level of cost optimisation at the pod level, rather than 2 at the pod and VM level.

Yeah, I wouldn‘t do it

<3s is our target 95% is <2.5s. We have a keep warm function that always keeps 5x "warm"-ish so we rarely hit cold starts. Keep warm is not fool proof as AWS got smart about people constantly doing keep warms, but it definitely helps.

I had no idea you could do this. I’m going to migrate my personal django projects onto lambda this week.

+1, take a look at Mangum: https://github.com/jordaneremieff/mangum/

What’s the process / worker model when it’s run like that? Is there a continuous process running listening on a port waiting for a request?

That's a very fat function to run on Lambda. What's the advantage compared to run the same container in Fargate? Compared to a VM I guess you're saving the sysadmin time.

Do you have some recurring activities to babysit Lambda or is it zero maintenance for your team?

Best thing about Lambda it is typically zero maintenance.

What’s your QPS like and what’s the value-add over running it on something like EC2? (or AppEngine etc if you want something more managed?)

Quite low. We're a b2b application, so each customer is worth a lot but very very underutilized CPU and relatively few customers.

Main advantages: Server $$ significantly lower than CI/CD; Bottleneck becomes DB, can have 1000x instances no problem (but never use more than 5x). No EC2 to manage just deploy the docker.

How is that different from just a normal Webhoster? Except, being more expensive, I guess?

It gives you the flexibility to easily swap in pieces in another language without making architectural changes while being as simple to work with (after devops work) as a monolith. If you have an orchestration setup to manage your functions rather than using Lambda/etc it doesn't cost any more than a monolith either (aside from competent devops salary).

> It gives you the flexibility to easily swap in pieces in another language

Which you can also do with a Webhoster. Well, it depends a bit on what languages or environment they offer of course.

> without making architectural changes

I guess those pieces have a way to communicate with each other? So the point here is more that you are forced to work with an architecture which is from the beginning designed for pluggable pieces.

I should've put context, but we are a b2b and barely have any CPU utilization and mostly in bursts.

Normally we would have at minimum 2x Fargate "mediums" sitting around for redundancy. ~US$130/mth now our server costs <$5/mth. Our CI/CD is WAY more expensive.

For me, serverless solved a problem at the opposite side of the spectrum of “scalability”. It helped me to start new projects very easily.

Since I like React and the Next framework, Vercel is a one-stop shop for me now. With their new storage feature, I can start creating a fullstack web app very easily. Frontend, functions, and Postgres, all hosted one Vercel and very easily integrated.

Having a Postgres DB makes it very easy to export to a proper server in the future, if I need it

We cut one system’s cloud bill by two thirds by moving to virtual private servers. The amount of time/effort serverless and other x-as-a-services were saving in maintenance was not worth the cost and other constraints they caused.

*Why The Someone Else's Server Revolution Has Stalled

I always wonder about this scalability part:

>resources used on serverless frameworks are typically paid for by the minute (or even by the second). This means that clients only pay for the time they are actually running code. This contrasts favorably with the traditional cloud-based virtual machine, where often you end up paying for a machine that sits idle much of the time.

I'm going to ask something very stupid because I'm very ignorant but... when do you need this?

In my mind you have scenario 1: the app never runs because no users, so running it 24/7 is a waste. In this case why cloud? Pay $5 a month for a simple web host?

Scenario 2: you are hitting 100% on a server and you need a second server but that second server will only use 1% most of the time. Again, why not just upgrade the server with your web host?

Scenario 3: you have a bajillion users on peak hours and a tenth of a bajillion on other hours. I assume you would have a business model that affords you to hire someone to manage scaling your infrastructure?

What am I missing in this picture?

It stalled the second people realised the actual costs are significantly north of using your own ifrastructure, and is often burried in a line like "oh by the way you'll need a NAS Gateway and it'll cost you an aditional $50k a month"

I think the revolution happened and here we are.

I guess when this was written (3+ years ago) a lot of people were still high on the marketing hype so the reality seemed flat.

But I think we got some nice options.

(Which, of course, aren't silver bullets that solve all the problems.)

I think appengine is one of the most underrated "serverless" offerings out there. I deploy everything on it, with almost zero concern about infra or server architecture

And cost? How does that factor in to your use case?

My use cases are probably regarded as small/medium, which need a handful of instances running, which you can effectively scale down to zero with Googles free tier (basically allows you go goto 1 instance for free), so cost has not been a major consideration.

When did go fast break things become go fast be stupid.

Serverless is like driverless.... Sure you might be in the backseat but there is a chauffeur up front and you pay a lot of money for that.

Unless your not paying for it (directly) --- serverless makes a LOT of sense if you have light weight tasks that can be distributed and you send it to "excess" infrastructure you already own/run.

While I see use cases for serverless, Simon Wardley is so bullish on it that it makes me wonder if it's a bigger deal than I'm making it out to be. For example: https://twitter.com/swardley/status/1376842878182428672

most critiques do not apply if you look at providers that have open source runtimes. for services running on deno or workerd/cloudlfare workers you can add your own runners on big cost effective servers that allow longer execution and serve core locations or certain workloads with the same code. "limited languages" still applies as these are limited to js/wasm languages. "not being able to run entire applications" still applies only if you include the database servers, as these will need special operation. (but running databases was most of the time a seperate thing from the core application anyways)

There is varying levels of serverless.

I think FaaS turned out to be a largely bad idea.

Stuff like Fargate, GKE Autopilot and Cloud Run I would also consider "serverless" just in a weaker sense. I can see these platforms having pretty wide appeal for workloads that don't benefit from stronger control over the underlying VMs and networking.

The biggest selling point that seems somewhat legitimate is a bunch of these have scale to zero capability. This is pretty useful for on-demand code that is required very infrequently and for which cold-starts are a non-issue.

I think where they fall down is that once you run up against one of their hard constraints is that the cost of moving to something without those constraints acts as a high enough activation energy to instead contort the application architecture to work around said constraints. This I feel is very bad and leads to horrible results, especially when aforementioned platform is FaaS.

In practice if you know you are going to want to run any non-trivial long-running or stateful services you are just better off going k8s from the start. The API is good, the managed options are good, life is just less complicated when you don't need to deal with proprietary bullshit.

>The biggest selling point that seems somewhat legitimate is a bunch of these have scale to zero capability. This is pretty useful for on-demand code that is required very infrequently and for which cold-starts are a non-issue.

I don't quite understand this. You can pay for 1 physical/virtual server and have apache serve 2 different domain names, right? Can't you just drop your infrequent app there so it shares resources with, I don't know, your more regularly accessed website?

The only scenario I can imagine this would be useful is if you own zero servers or the code is incompatible with the server you do own for some reason.

Well you do understand it, enough to work out how to live without it. :P

That said there are some key things a scale to zero serverless app has that solution does not.

1. Isolation. Your infrequent function may very well be infrequent but it's execution could then impede other code executing on that box (if it's computationally or I/O intensive for instance), or expose other code on the box to security vulnerabilities.

2. Independence. Similar to above but more addressing logical segregation. Once you are sharing a server or say piggybacking on a monolith as an extra endpoint there is now a lot of conflated concerns. For instance auto-scaling algorithms etc may not work well with the sort of intermittent/infrequent execution and it would be much better living in it's own context for that reason. Other things that end up all jumbled up: Cost accounting, IaC/deployment/CI/CD, ownership, observability, etc.

Generally all of this still isn't enough to justify serverless but sometimes it could be... but in most of those cases just eating the cost of 1 replica and autoscaling on k8s is usually better.

Thanks for the explanation, I really appreciate it!

Salesforce and its Force.com PaaS is another "serverless" offering with large uptake, Force.com (Apex programming language) was first offered in 2007.

I hate Azure functions. It's the weirdest shit I ever had to work with and the fact that most documentation relies on clickety click examples on vscode or some javascript tool I have no access through my build and deployment environment infuriates me more. Please for the love of god give me a proper server.

Deno runtime is a delight to work with.

... because serverless still has a server, which is controlled by a 3rd party?

The serverless name is unhelpful. What these companies really mean by serverless is:

"your code that handles HTTP requests must be able to startup, run, and shutdown effectively instantly"

aka "serve your HTTP without bloated long running processes."

aka `cgi-bin` scripts.

I think it means you remain serverless.

(2020)

Added. Thanks!

TLDR: serverless is interesting to whoever won't pull out an Excel spreadsheet and compare costs.

Reply on news.ycombinator.com