Serverless: I'm a Big Kid Now
simplethread.com
simplethread.com
Actually, I am setting up a serverless app now. 4-5 lambdas, s3 buckets, RDS, IAM roles, and 6 weeks (easily) getting everything into CFT's and Ansible so that I can deploy this relatively small app.
You know how I would replace all that stuff? 1 single VM. (Alright, maybe 2, 1 for the database.)
A server buys you ease of deployment. It really does. Code needs an environment. Lambdas punt on dependency management and environment. You have to build the environment for the lambdas using IAM roles, RDS, s3 -- out of toothpicks. All that stuff (security, storage) is already there for you in a VM.
I was talking with my brother in law about his serverless deployment of ~23 microservices he had carved up out of a Java Spring app. A small change anywhere in the code meant he had to redeploy all of it, not because of how the code was structure but that was the only way the build tools allowed him to do it. More problematically, the dependencies had to be built into every single lambda and made the build longer. (No, lambda layers didn't work for us. Way too complicated to figure it out.)
Honestly after going through tons and tons of pain to deploy these lambdas in both cases, it's just way easier to deploy on a VM. Maybe not for developers who just can't do anything if it's not done in their IDE, but from a broader, more globally-looking DevOps perspective, there are hidden costs.
Yes, it's easy to develop for lambda. But WOW, it's awful in deployment in comparison to just putting things on VMs.
Now, could a lot of serverless deployments do just fine on a single VM because they never get enough traffic? ...probably. To my mind that's the real problem with serverless: it's a premature optimisation that simply isn't needed in a lot of cases.
But what I see a lot of is engineering teams making sweeping consistency judgements. RDS and S3 are great as services, everything should be a (micro)service. Our calendar feels amazing as a React SPA—everything should be built in React.
No. 90% of a product is just CRUD to support the 10% of magic that customers really value. Ship the 90% as a monolithic CRUD app and spend those cycles on that 10%.
It’s been many years since then and all this really did was bloat the client with metric tons of business logic, punch inconsistent holes through the API layer, and cause the entire web product to have abysmal load times (normal is somewhere in the 5 second range IIRC).
I think it’s understated how much damage the young technical leaders typically employed by early stage startups can really do. I can tell you this because I was exactly this person, and I got to see what following hype without doing actual mature research leads to.
For a new project, serverless seems to involve a ton of momentum-stealing upfront work before you can really get going, and thereafter you do always feel one step further removed from your code, so it can be a pain to debug and troubleshoot.
OTOH, for any real production application, a single VM instance is rarely sufficient if it's critical to the business, and even within a single machine there always seem to be a lot of ops tasks like setting up process keep-alive stuff, rotating and deleting logs, making backups, etc.
For now, either approach can work great. Long term I think serverless is the way to go, but it still has a lot of maturing to do (the good news is that pretty much all of the annoyances of serverless seem like very solvable problems).
This reminded me of a story I heard about RMS's kernel, GNU Hurd. Linux being built as a monolith in the 90's, Hurd was being built as a microkernel (microservices) architecture. Guess which one was easier to debug[1]?
1: https://biblioweb.sindominio.net/telematica/open-sources-htm...
I find it makes sense more for my uses cases to use it as a tool if I need to extend a monolithic app, not as a standard to create an app.
Those aren't microservices, by definition.
Pretty sure that's the point OP was trying to make - they put in the effort to build microservices, and crappy tooling means they are instead stuck with an mangled over-engineered monolith
Regardless, it doesn't sound like microservices, it sounds like a distributed monolith.
Having built a product out of microservices and lambdas I definitely have not found that build tooling has ever forced a full redeploy. I suspect this is just a case of converting a monolith to a microservice being much more difficult than people realize.
Yea, you are probably right on the money.
IME lots of folks struggle with DRY when trying to actually implement their services. In teams I work with, we frequently see attempts to build a shared library and have to re-hash the discussion every time
If your application does not meet any of those requirements then call it something else, as it clearly isn't a microservice.
Not sure you read my post. I called it a "mangled over-engineered monolith"
What build tools? We have over 60 Lambdas and aren't aware of any issues like this. We literally had to build extra tooling to be able to push more than one Lambda simultaneously so I'm a bit confused how is this related to the platform.
The only "build tools" Lambda itself has is literally a file upload and a version promotion.
> Actually, I am setting up a serverless app now. 4-5 lambdas, s3 buckets, RDS, IAM roles, and 6 weeks (easily) getting everything into CFT's and Ansible so that I can deploy this relatively small app.
I'm sorry but if this took you 6 weeks then you're doing it wrong. To be fair, I haven't tried using CFTs and Ansible for my serverless deployments, but then again these seemed like big time overkill to me. Your experience seems to back that up.
Look, I don't want to say infrastructure-as-code is a bad thing, but don't blame the infrastructure when it's your choice of tools that is the problem. The AWS CLI makes it so easy to write a bash script to deploy a small project. But hey if you think an extra 6 weeks is worth it to use Ansible, then by all means...
I now work in a company where they do things themselves, bare metal, and use ansible, ssh, etc pretty much basic stuffs.
I like this way more than my AWS/kubernetes insanity that my previous company had. It was so hard to debug, and everything needs a ticket.
I've heard talks by experienced K8s adopters (I want to say "Kubernetes failure stories", but googling comes up with similar hits but not the specific talk I was thinking of) where they mention that when K8s goes bad, it has all the negatives or knowing the "traditional" tech stack plus all the failure modes of K8s itself. They argue that it's more stuff you have to know about, not less; and when trouble hits, it can get very complicated to understand why.
(Not picking specifically on K8s; this applies to similar orchestrators/container tech).
1: https://medium.com/omio-engineering/cpu-limits-and-aggressiv...
Sure, but k8s is big enough that "very few people know what they're doing" is a valid argument. It's loosely like saying that git sucks because the technology is solid but it's extremely complex and hard to get a grasp on.
The article to which I linked describes itself as "a wild ride of discovery" but all that has been discovered is basic aspects of the Linux process scheduler. The author could have skipped the wild ride if they had understood Linux first.
Still not the exact talk, but I think it's from the same guys. I remember they explained how with K8s, the "surface area" of problems grows almost uncontrollably. You have all the problems of doing things manually (or at least, when things go wrong you must have a very good grasp of them), plus all the problems of K8s.
(Do note they are not arguing against K8s, just pointing out its multiple pitfalls)
Ultimately I think it's just another leaky abstraction like so many other things we do. Wonderful when it works as intended and less so when it goes wrong.
I also think that the more layers of software and management you add in makes it more complicated to troubleshoot, you simply get a lot more to learn in order to understand things.
One of my life philosophies is "It's always more complicated", meaning when you think you understand something you've barely just started scratching the surface of the complexity of it.
I can understand the pull of the container and lambda style workflows, you gain feature velocity which is ultimately what gains you the business. On the flip side, I also suspect that it builds technical debt faster.
Besides this, the other service I like is Fly https://fly.io/ with lets me submit a container and runs that all over the world where necessary. Google Cloud Run does something similar, but its region locked. Fly is like global lambda for containers.
Between these tools and services, I’ve stopped caring about servers completely. Never need to install a package or use ssh again. I either submit just the app executable to lambda using Up (for light HTTP APIs) or send a container to Fly (for heavy servers, TCP, web sockets) and that’s it.
Honestly, I don't think there's anything that compares at this price point and ease of use.
IMO I don't think Heroku and others fit the serverless paradigm.
To me serverless is about automatic scaling of performance and cost.
With Heroku you need to provision capacity in advance, and you pay for it whether you use it or not. Heroku doesn't scale automatically either, you do that manually.
But I don't know much about their pricing.
Netlify does definitely fit the serverless paradigm.
> Serverless container services such as Heroku, ... AWS ECS/EKS Fargate, Google Kubernetes Engine, and Azure Kubernetes Service provide you with an environment where you can build a container
Not building or managing a container is why you're using serverless.
You don't have to worry about it, you get to worry about it.
At least from a cost perspective, GKE isn't serverless. Also I always end up with 2 node pools because the default one is misconfigured. Also kube runs so much crap that the first node barely fits any app containers.
Wouldn't call GKE a win for 'not managing nodes'.
Is this true? I always thought things like Docker are massively insecure because they don't respond to the threat landscape that well, since they are kind of 'frozen in time' and kept that way for years at a time without any critical security updates.
For those who want to get some fast insight to technology which is being adopted or dropped across industries in the software space then definitely check out Thoughtworks Technology Radar. It's regularly updated.
There's also many reasons given that don't really make sense.
> Your containers are cattle, not pets. If your container crashes, a new one is automatically fired up.
If my container crashes, I want to find out why so I can prevent it from happening again. One of the issues with these serverless technologies is that they make this kind of debugging harder. I don't want code to randomly crash and just getting restarted to be the solution. It means there is something wrong. To be fair he does mention this later on in the article.
> Serverless functions force you to write your code in a stateless way
So what. If I want to have my own server I can also do it stateless if that makes sense to me. Being forced in a direction is not a positive.
I think the scaling is very interesting. But this is not something that's required for many applications. It doesn't mean everything should be serverless. It just means there's a good option to choose from for usecases that can really benefit from it.
Pushing serverless for everything is like putting blockchain into everything because it's a buzzword right now.
It depends on your work load. And your trust level in Amazon of course.
It's expensive to keep that server maintained.
rpm -qa | grep -i kernel && yum -y update && reboot && ..... After system comes back online, ensure the kernel is a newer version rpm -qa | grep -i kernel
Other utilities like ukuu for debian or ubuntu systems make updating the kernel a breeze.
If you're worried about specific packages, sure that can be daunting, so avoid updating software you are concerned about breaking ( keep an eye on security alerts for that software, which you should be doing anyway if you are using it on a VM container, or serverless... none are immune to vulnerabilities! ) and do not install unnecessary software.
I'll take issue with this one. I'm fully aware I'm weird, but I do actually enjoy managing servers. And with that effort spent managing servers, I buy the ability to actually create the environment in which my code runs, and interact and improve it over time, rather than relying on someone else's idea of what's good for my code. Moreover, I get the ability to dictate the security environment in which my code runs, instead of hoping my cloud provider is doing it properly enough for me (and not busy snooping on what I'm doing in the process).
I think what the poster means is "I don't want to manage servers, I want to write code." Which: fine. The world needs people who focus on writing good code. But I'm sick and tired of this attitude that coding is somehow more noble or better than the important work of building and maintaining the environment in which the code runs. You don't like it? Awesome, don't do it. But stop spitting on people who do, and who legitimately enjoy doing it.
S3, DynamoDB, AppSync, API-Gateway, and Faregate are also serverless, but not FaaS products.
If you want to perform actions that might be throttled (cloudwatch for example) and are not critical for the response, you cannot have a singleton processing them after you return, because they might not be processed.