Serverless: slower and more expensive
einaregilsson.com
einaregilsson.com
1. Don’t use .NET, it has terrible startup time. Lambda is all about zero-cost horizontal scaling, but that doesn’t work if your runtime takes 100 ms+ to initialize. The only valid options for performance sensitive functions are JS, Python and Go.
2. Use managed services whenever possible. You should never handle a login event in Lambda, there is Cognito for that.
3. Think in events instead of REST actions. Think about which events have to hit your API, what can be directly processed by managed services or handled by you at the edge. Eg. never upload an image through a Lamdba function, instead upload it directly to S3 via a signed URL and then have S3 emit a change event to trigger downstream processing.
4. Use GraphQL to pool API requests from the front end.
5. Websockets are cheaper for high throughput APIs.
6. Make extensive use of caching. A request that can be served from cache should never hit Lambda.
7. Always factor in labor savings, especially devops.
If "serverless heros" are running around promoting Lambda, newcomers will use it without thinking twice...
https://github.com/awslabs/aws-lambda-rust-runtime
https://crates.io/crates/lambda_runtime
you rock! That docker based build system makes building those MUSL based rust binaries a snap!
Can you show some empirical evidence that supports this? In my experience this is another nebulous serverless hype claim that doesn't withstand scrutiny.
No. There's nothing common sense about it. It only seems plausible if you read the sales brochure from a cloud vendor and have no experience with all the weird and whacky failure modes of these systems and the fact that none of the major selling points of serverless actually work as advertised unless you dedicate significant engineering time to make them work - as the GP comment has demonstrated. The amount of engineering time required to make serverless work quickly catches up to or even exceeds just doing the damn thing the normal way.
And that engineering time is not transferable to any other cloud vendor, and neither is your solution now. So congratulations you just locked your business in.
Serverless only makes sense if you have a fairly trivial problem and operate on really narrow margins where you need your infra and associated costs to scale up/down infinitely.
That’s exactly the point. The web application needs of most startups are fairly trivial and best supported by a serverless stack. Put it another way: If your best choice was Rails or Django 10 years ago, then it’s serverless today.
As a side note, I've learned that the existence of a large number of viable ways to accomplish a task is a pretty big anti-signal for the desirability of accomplishing that task in the first place. When I started my career in 2000, there was a huge debate over whether the "right" way to develop a desktop application was MFC or .NET or Java or Visual Basic or Qt or WxWindows. The real answer was "don't develop desktop apps, because the web's about to take over". When the big web 2.0 businesses were founded from 2005-2011, there were basically two viable options for building a webapp: Rails or Django. Now that everyone's arguing about microservices vs. Docker vs. Kubernetes vs. serverless vs. Beanstalk vs. Heroku, it's likely that the real answer is "develop a blockchain app instead".
That's... not true. The choice of web stack – and, in fact, the whole software – is just a piece of what a startup may need.
Seriously, look at the list of YC startups on 2018 and tell me if most couldn't use either something like Rails, or a Single Page App In React With A Serverless Backend. And it wouldn't matter one bit.
https://techcrunch.com/2018/03/20/these-are-the-64-startups-...
> it's likely that the real answer is "develop a blockchain app instead".
I hope that was sarcasm.
Pretty subjective statements, I suppose we don't have the same definition of "trivial".
> If your best choice was Rails or Django 10 years ago, then it’s serverless today.
Comparing the features of Rails or Django with serverless is like comparing a spaceship with a skateboard.
Well, not necessarily? This assumes that the implementation is sound but it is not at all uncommon for abstractions to leak which end up causing more pain than they solve
It is not at all obvious what it just can't be used for. In our case, Julia.
Software professionals often sees benefits without understanding the tradeoffs.
- iRobot (maker of Roomba) has been running its entire IoT stack serverless since 2016 (architect Ben Kehoe is a worthwhile follow on Twitter)
- Reddit’s video hosting service is built and operated by a single engineer on a serverless stack
Shouldn't this not be a problem if you're doing 10 million requests a day? If you have enough requests, your lambdas should stay hot most if not all the time.
Then if this is false, in your Application_Start() you can set this: TelemetryConfiguration.Active.DisableTelemetry = true;
Came here to post this and agree 100%. Moving to Serverless requires evaluating the entire stack, including the server side language choice, and how the client handles API calls.
Often a move to serverless is better accomplished gradually in stages than the quick "lift and shift" that AWS like to talk about so much. Sometimes you can simply plop your existing app down in Lambdas and it runs just fine, but this is the exception not the rule.
At this point in time the only places I’d use lambda are for low-volume services with few dependencies where I don’t particularly care about p99 latency, or DevOps-related internal triggered jobs.
We migrated our company webapp to Heroku last year. We pay about 8 times what a dedicated server would cost, even though a dedicated server would do the job just fine. And often times, people tell me "Heroku is so expensive, why don't you do it yourself? Why pay twice the price of AWS for a VM?"
But the Heroku servers are auto-patched, I get HA without any extra work, the firewall is setup automatically, I can scale up or down as needed for load testing, I get some metrics out of the box, easy access to addons with a single bill, I can upgrade my app language version as needed, I can combine multiple buildpacks to take care of all the components in our stack, build artifacts are cached the right way, it integrates with our CI tool, etc, etc.
If I had to do all of this by hand, I would spend hours, which would cost my company way more. In fact, I'd probably need to setup a Kubernetes cluster if I wanted similar flexibility. By that point, I'd probably be working full-time on devops.
At my previous company we had project with an AWS deploy process that only two developers could confidently use. Teaching a new developer & keeping them up to date was a big time sink.
For comparison we had a Rails app setup on heroku that on day one junior devs were happily deploying to (plus we had Review apps for each PR!)
The big difference we are migrating away from Heroku to Kubernetes for the same reason.
- Corrupted build? Reverse button for the rescue.
- SSL? They got you.
- Adding new apps in less than 1m?
and so on ...
Granted that it does impose some limitations, and therefore isn't right for all apps. But it does seem like it would work for a large percentage of web apps and REST api's.
I guess the reasoning would be that this way the actual time spent in serverless code is shorter and by proxy the service becomes cheaper?
With custom runtimes that's not the case anymore. I write my lambdas in Rust.
Can't stress (7) enough, would also add 'morale' savings. It can be really stressful for developers to deal with gratuitous ops work.
Granted, this approach is much lighter than uploading an image directly.
I always sorta assumed that Amazon pre-initialized runtimes and processes and started the lambdas from essentially a core dump. Is there some reason they don't do this, aside from laziness and a desire to bill you for the time spent starting a JVM or CLR? Does anyone else do this?
1. not use the programming language that works best for your problem, but the programming language that works best with your orchestration system
2. lock yourself into managed services wherever possible
3. choose your api design style based on your orchestration system instead of your application.
4. Use a specific frontend rpc library because why not.
...
I've hacked a few lambdas together but never dug deep, so I have very little experience, but these points seem somewhat ridiculous.
Maybe I'm behind the times but I always thought these sort of decisions should be made based on your use case.
EDIT: line breaks.
Then why comment? You clearly don't understand the use-case that AWS fits.
I've had jobs that took 18 hours to run on single machine finish in 12 minutes on Lambda. I could run that process 4 times a month and still stay within AWS's free tier limits.
For the right workloads it is 100% worth realigning your code to fit the stack.
Because the instruction list from above isn't backed with any solid reasoning and because commenting is what people do on HN.
>You clearly don't understand the use-case that AWS fits.
Pray tell, what is this enviable use case that I so clearly do not grasp?
Ok I'll bite. What takes 18 hours to run on a single machine but finishes in 12 minutes on Lambda.
You'd need some logic to shut down the instances once it's done, but the simplest logic would be to have the software do a self-destruct on the EC2 VM if it's unable to pull a video to process for X time, where X is something sensible like 5 minutes.
Doing that in a series took forever. It took something like 14 hours to generate the screenshots, then 4 hours to upload them all. Spreading that load across lambda functions allowed us to basically run the job in parallel. Each individual lambda process took longer to generate a screenshot than on our initial desktop process, but the overall process was dramatically faster.
This argument is basically: no counterpoint to the original post, but you can do things that are also easy on any other comparable platform.
Tell me again what I don’t understand?
As someone who has done both, it's far, far easier to stand up a lambda than it is to manage a cluster of servers.
In my mind the thing that makes lambda “easier” is they make a bunch of decisions for you, for better or worse. For real applications probably for the worse. If you have the knowledge to make those decisions for yourself you’re probably better off doing that.
The whole value proposition behind AWS is that they can do it better than your business due (directly or indirectly) to economies of scale. I think Kubernetes is super cool, but rebuilding AWS on top of Kubernetes is not cost effective for most companies--they're better off using AWS-managed offerings. Of course, you can mix and match via EKS or similar, but there are lots of gotchas there as well (how do I integrate Kubernetes' permissions model with IAM? how do I get Kubernetes logs into CloudWatch? how do I use CloudWatch to monitor Kubernetes events? etc).
Not if you already used your free tier. Lambda is X free per month. Free tier t2.micro is X free for first 12 months.
Now that guy's workload doubled and needs another colleague to be able to deliver.
Optimistically, it could represent a positive trade-off that replaces perpetual upkeep with upfront effort, and all-hours patching and on-call with 9-5 coding.
In practice, I think a lot of those fixed costs get paid too often to ever come out ahead, especially since ops effort is often per-server or per-cluster. The added dev effort is probably a fixed or scaling cost per feature, and if code changes fast enough then a slower development workflow is a far bigger cost than trickier upkeep.
Moving off-hours work into predictable, on-hours work is an improvement even at equal times, but I'm not sure how much it actually happens. Outages still happen, and I'm not sure serverless saves much less out-of-hours ops time compared to something like Kubernetes.
You’d simply upload an ear or ear and the server and deployed would handle configuration like db etc.
It worked perfectly (ear/war, the persistence framework was too verbose and high level imo, but that was replaced by hibernate/jpa). There was too much configuration in xml, but that could easily be replaced my convention, annotations and some config
Again.. we are running in circles, and this industry will never learn, because most “senior” people haven’t been around long enough.
And that likely won't change in our lifetime, given the rate of growth in demand for software: we literally can't create senior engineers fast enough for there to be enough to go around.
As an aside, I have the privilege of working with a couple of senior folks right now in my current gig, and it's pretty fucking fantastic.
What's the biggest idea you've seen abandoned and then reinvented with a new name?
The worst for me is the vendor lock in, directly followed by the costs.
- OS version patching, patch windows & outages, change mgmt relating to patching and reporting relating to OS patching
- antivirus, the same patching management and reporting as above
- intrusion detection / protection, host based firewalls, the same patching and management as above
- Other agents (instance health monitoring, CMDB, ...)
- Putting all this junk on a clean OS image any time something changes, re-baking and regression testing everything
This all adds up, and can be a significant cost to an organisation - team(s), licenses, management, etc.
The only thing I'm sad about is that you can only host it on Azure. I'd love an open standard for function-like programming and hosting.
Of course, not everything is suitable for this model, and it certainly won't be the most cost-effective or performant way to do it for everything either.
(If Reddit’s video hosting being built and operated on a serverless stack by a single a engineer won’t convince you, I don’t know what will.)
I even tried to download them with youtube-dl but it doesn't work.
That thing almost never works.
And of course, baremetal provides a much better ROI than VMs/VPS/<insert glorified chroot/jail here>.
The comment is saying that Lambda has limitations and works best when considering those limitations. If those limitations don't fit your use case, you shouldn't be using Lambdas - or, at least, don't expect it to be an optimal solution.
What does this look like in practice? Doesn't this increase response time for the initial requester?
DevOps is not a synonym for "ops" or "sysadmin". It's not a position. DevOps is like Agile or Lean: it's a general method with lots of different pieces you use to improve the process of developing and supporting products among the different members of multiple teams. DevOps helps you save money, not the reverse.
You don't even need an "ops person" to do DevOps.
Furthermore, eople like to compare runtime startup times, but this tells a very small portion of the story. For most applications, the dominant startup cost isn't the startup of the runtime itself, but the cost of loading code the app code into the runtime. Your node.js runtime has to load, parse, compile, and execute every single line of code used in your app, for instance, including all third-party dependencies.
Compare, for instance, the startup cost of a "hello world" node.js function with one that includes the AWS SDK. At least, six years ago, the Node.js AWS SDK wasn't optimized at all for startup and it caused a huge (10x?) spike in startup time because it loaded the entire library.
I would argue that the only languages that are a really good fit for Lambda are ones that compile to native code, like GoLang, Rust, and C/C++. The cost to load code for these applications is a single mmap() call by the OS per binary and shared library, followed by the time to actually load the bytes from disk. It doesn't get much faster than that.
Once you've switched to native code, your next problem is that Lambda has to download your code zip file as part of startup. I don't know how good Lambda has gotten at speeding that part up.
Doesn't sound like your point number 1 is valid at all, quite the opposite.
I can think of a number of other languages that would probably easily surpass these, especially on latency.
"Serverless" is not a replacement for cloud VMs/containers. Migrating your Rails/Express/Flask/.Net/whatever stack over to Lambda/API Gateway is not going to improve performance or costs.
You really have to architect your app from the ground-up for serverless by designing single-responsibility microservices that run in separate lambdas, building a heavy javascript front-end in your favorite framework (React/Ember/Amber/etc), and taking advantage of every service you can (Cognito, AppSync, S3, Cloudfront, API Gateway, etc) to eliminate the need for a web framework.
I have been experimenting with this approach lately and have been having some success with it, deploying relatively complex, reliable, scalable web services that I can support as a one-man show.
At least I don't have to learn that complex "systems admin" stuff.
A serverless solution that eliminates the Web framework (and thus the stack in which is being run) does most of that for you, at the expense of extra cost or infrastructure deployment complexity, but once it is done, scaling and maintenance are easier.
As a hobbyist, I wouldn't have the time or motivation to completely rewrite a project if that happened, which would be necessary since a Firebase app (like a heavily AWS-integrated serverless app) is not just technologically but architecturally tied to that environment.
First and foremost it is client side SDK's (web, mobile) for their database products, their newest being Firestore that provides better query capabilities compared to their original Firebase Realtime database (while still offering real-time capabilities).
Along with that is Firebase Authentication, which manages user accounts and authentication.
The real magic comes in with Cloud Functions (their version of Lambda) which allows for hooks in to all sorts of events occurring from usage of these database and authentication products (and other cloud services).
Hook into database writes, updates, deletes, user creation, Google Cloud's pub-sub events and many more. They also offer static website hosting as well as hooking website serving into cloud functions (for server side code execution).
In the context of a website, all of these work together to allow for usage of the JAMstack[0] architecture which decreases your infrastructure resources you need to manage and cost.
Everything you need for a web or native app, and it's all integrated together rather well.
It's entirely missing the point. At the end of the day, you have to look at your specific usage pattern and pick the best option for you. Obviously, as with any other technology, anyone who forces a specific option in every possible situation is most likely wrong.
Colleague: Lambda is awesome, we can scale down to zero and lower costs! We love it! We use cool tech!!
Me: What did you do about JVM warm up?
Colleague: We solved it by having a keepalive daemon which pings the service to keep it always warmed up.
... Me thinking: Uhh, but what about scale down to zero?
Me: What do you do about pricing when your service grows?
Colleague: We use it only for small services.
... Me thinking: Uhh, start small and STAY SMALL?
Me: How was performance?
Colleague: It was on average 100ms slower than a regular service, but it was OK since it was a small service anyway.
... Me thinking: Uhh, but what about services who _depend_ on this small service, who now have additional 100ms times to comprehend with?
Overall, I think his answers were self explanatory. Lambda seems to be a fast prototyping tool. When your service grows, it's time to think how to get out.
My thoughts EXACTLY. The great power in "serverless" architecture (i.e. AWS Lambda + AWS RDS + AWS Gateway) is how it empowers prototyping a new product.
Counterintuitively, it's future-proofing. You should know in advance that it's too slow & expensive. But you get to spin up a prototype backend very rapidly, pay only for what you're using while prototyping, and Lambda's inherent limitations force devs to build modularly, start simple, & stay focused on the product's main goals.
When the time comes to need scale, either at launch or even later when user count goes up, your "serverless" backend can be relatively easily replaced with servers. Then, just like that, in response to scale need your product's costs and response time go down instead of up.
It's a nice way to build a software product: rapid prototyping plus easy future cost decreases built-in.
Which is wrong and can be attributed to bad serverless evangelism in the past.
Serverless is building your system with managed services and only drop-in a FaaS here and there when you need some special custom behavior.
See how far you come with AppSync, Firestore or FaunaDB. Throw in 0Auth or Cognito and then when you hit a wall, make it work with FaaS.
Can't you just do something on your local machine?
There's stuff like dotnet new for .NET where I can just run that and have a skeleton project for a backend and I can start writing code immediately. I assume there's template creators for other languages as well.
For example, I have a few data scrapers written in JavaScript but my regular stack is lamp.
So I don't have any need to run a node server 24x7 just for those once a day tasks.
But I have even found myself not needing serverless for that because everything is running in a kubernetes cluster. So I can just setup a cron to run them which launches the needed node containers.
So I guess in effect, I am just using a sort of self-managed "serverless".
The serverless proponents are selling their paradigm as simple solution, which leads many people to believe simple means FaaS.
Throwing Lambda on all backend problems is a setup for failure. Often transfer and simple transform of data can be done serverless without a Lambda, which cuts costs AND leads to better performance.
I’d be curious about how much memory/cpu was allocated in your experience and the OPs, there’s nothing magical about lambda to make it slow.
This does happen. We have a serverless API forwarding service on Azure that was designed to simply format and forward calls from a vendor. We know the volume, there will not be any surprises, and it is immensely profitable over the old solution to the tune of thousands of dollars per day. Our use case is probably pretty uncommon, however.
The 100ms extra time is nothing. I mean - are you trying to solve at Google or Amazon scale?
I run simple Lambdas that read from some SNS topics, apply some transforms and add metadata to the message, and route it somewhere else. I get bursts of traffic at specific peak times. That’s the use case and it works well. The annoying part is Cloud Formation templates but that’s another topic.
Let’s say I’m processing messages off a queue. P90 @ 50ms vs p90 at 100ms doesn’t necessarily make a difference. What are my downstream dependencies? What difference does it make to them?
At the end of the day, value is what you care about - not necessarily chasing a metric because lower is absolutely better. What’s the cost of another x milliseconds of latency considering other trade offs (on going operational burden, extensibility, simplicity, scalability, ease of support etc etc).
If 50 ms latency means I can have a solution that can auto scale to massive spikes in traffic due to seasonality or time of day vs a reduction of that latency but I have to spend time capacity planning hardware and potentially holding extra capacity “just in case”, then again, optimizing for a single metric is pointless.
It's obviously at a different point in the stack, but the promise is similar: "Just say what you want and it magically happens" — and indeed that's the case for FaaS when your problem is small enough. But XML-configured frameworks also offered that convenience for their "happy problem space". You could get things done quickly as long as you were within the guard rails, but as soon as you stepped outside, the difficulty exploded.
I'm not convinced AWS Lambda is all that different from a web framework, it's just on a higher level. Instead of threads responding to requests, you have these opaque execution instances that hopefully will be spun up in time. Instead of XML files, you have a dense forest of AWS-specific APIs that hold your configuration. That's how it looks from the outside anyway.
The promise of serverless is pretty simple, and pretty useful for the right use case - be it unpredictable load, or just very low load, or very frequent deployments, or pricing segmentation, or you don't have anyone as DevOps, and so on and so forth.
I don't recall anyone saying there's any magic involved. The premise is exactly same as cloud compute - you (possibly, depends on ABC) don't need to provision and babysit a server to perform some action in response to http request (or in case of aws lambda, other triggers as well).
I have had so many conversations with devops managers and developers who are individual contributors and the Lambda hype reached frothing levels at one point.
Contradictory requirements of scale down to zero, scale up infinitely with no cold starts, be cheap and no vendor lock in seemed to all be solved at the same time by Lambda.
Testability? Framework adoption? Stability? Industry Skills? Proven Architectures...? Are some of the other question marks I never heard a good answer for.
Serverless is specifically a stateless paradigm, making testing easier than persistent paradigms.
> Framework adoption?
Generally we use our own frameworks - I do wish people knew there was more than serverless.com. AWS throw up https://arc.codes at re:Invent, which is what I'm using and I generally like it.
> Stability? Industry Skills? Proven Architectures...?
These are all excellent questions. GAE, the original serverless platform, was around 2010 (edit: looks like 2008 https://en.wikipedia.org/wiki/Serverless_computing). Serverless isn't much younger than say, node.js and Rust are. There are patterns (like sharding longer jobs, backgrounding and assuming async operations, keeping lambdas warm without being charged etc) that need more attention. Come ask me to speak at your conference!
No because Lambdas are proprietary which means you can't run it in a CI or locally. Also, it becomes stateful if it pulls data from a database, S3 or anywhere else on AWS which it almost always does.
> Serverless isn't much younger than say, node.js and Rust are.
AWS Lambdas which I consider to be the first widely used Lambda service was was released in April 2015 which is 6 years after the release of Node.js. Also, Node.js is way more popular and mature than Lambda solutions.
Overall Lambdas are only useful for small, infrequent tasks like calling a remote procedure every day.
Otherwise, things like scheduling, logging, resource usage, volume and cost make Lambdas a bad choice compared to traditional VPSs / EC2.
Arc has a full sandbox, so yes you can run a lambda in a CI or locally. You should know this - if you're developing your apps by deploying every lambda as you edit it you're doing it wrong. Most lambdas don't really need you to simulate AWS.
> Also, it becomes stateful if it pulls data from a database, S3 or anywhere else on AWS which it almost always does.
Sure, persistence exists, but when people say 'stateless' they mean 'has no transient state'. They don't mean 'has no data'.
> AWS Lambdas which I consider to be the first widely used Lambda service
OK. You don't consider GAE (2008) widely used. I disagree - mainly because I was using GAE back then and thinking about this new world of no-memory, timeouts, etc - but lambda is definitely more popular.
Lambda is a function call. So it makes no difference if it’s proprietary or not.
Are you saying that it’s difficult to test passing an object to a function and asserting that it’s functioning as intended?
If I run a Node.js function in AWS Lambda, my Node.js version might be different, my dependencies might be different, the OS is different, the filesystem is different, so I or one of my node_modules might be able to write to /tmp but not elsewhere, etc.
It's the reason people started using Docker really. If you don't have the same environment, you can't call it reproducible or testable for that matter.
Note performance, acceptance, and monitoring modes
There’s a lot of annoying things about lambda. And a lot of stuff I wish was easier to find in documentation. But that doesn’t change the fact that Lambda is more or less passing an event object to your function and executing it.
Writing a function in node 12 and then running it on node 4 and throwing your hands in the air cos it didn’t work isn’t the fault of Lambda.
In any case, if you have a Node.js module or code with a native C/C++ build, that runs shell commands, that writes to disk (not allowed besides /tmp in Lambda) or makes assumptions about the OS, your "simple" function will absolutely return different results.
e.g: My lambda is called when somebody uploads an image and returns a resized and compressed version of it. This is done using Node.js and the mozjpeg module which is dependent on cjpeg which is built natively on install.
If I test my function on my machine and in Lambda it's very possible that I get different results.
Also, certain OSs like Alpine which are heavily used for Docker don't event use glibc as compiler, so again, another difference.
But isn't the idea of Docker that you can recreate the production environment? If you can't why use Docker in the first place?
Even if you did find a way, you would still need to keep it up to date in case AWS decides to update that environment.
Also, you compile before packaging, so you dev/CI system already has to be able to compile for Lambda, independenly from testing/debugging with Docker.
I saw that the SAM CLI uses an Alpine based image, does yours use Amazon Linux 2?
I'm jusz asking, because I compiled some libs on Cloud9 (which uses AL) and they worked on Lambda, so I assumed it's the same dist.
I don't use lambda but we have our jenkins spin up the same ec2 to run tests that we would spin up to do development so that we never run into this problem.
If you mean running a Docker container in Lambda, that is to may knowledge to possible. You could schedule Docker tasks in AWS ECS (their managed container service) but it's not meant for anything realtime and more for cron job type tasks.
If you mean emulating the Lambda environment in Docker, then I wrote an answer with the difficulties of doing that below to another user.
This is true, but it's not Lambda qua Lambda. That's just normal production vs. testing environment issues, with the same basic solutions.
Lambda may offer some minor additional hindrances vs. something like Docker, but I wouldn't consider that catastrophic.
You shouldn't be testing "on your machine" - that's the oldest excuse in the book!
You should build your function in a container based on AWS Linux, just the same as you should for a Lambda deploy. That guarantees you the same versions of software, packages, libraries, etc. It makes it possible for me to develop Lambda functions on a Mac and test binary-for-binary to the deployed version.
"Nothing you mentioned has anything to do with the ability to test a Lambda" is not ad-hominem, it's a statement of fact.
> It's great to see that factual evidence is answered with ad-hominem by the Lambda hype crowd.
I don't think that was a personal attack.
We've answered technical questions with technical answers.
- You have a definition of stateless which includes having no persistence layer, which is at best at odds with the industry.
- You think serverless was created with AWS Lambda which we've been kind about, but most people would say you're simply wrong.
- You're advocating for containers, which are well known for having their own hype as people write their own cloud providers on top of the cloud provider their employer pays for with dubious benefit.
Yes, in major only. Your lambda has node 10, you might be running 10.x or 10.y
> my dependencies might be different
No, the lockfile does that.
> the OS is different, the filesystem is different
Agree, but how much does one Linux+systemd different from other Linux+systemd? How much does the FS?
> It's the reason people started using Docker really.
VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless.
No, your lockfile doesn't care about build steps so any post-install script might run differently for the many other reasons listed.
> Agree, but how much does one Linux+systemd different from other Linux+systemd? How much does the FS?
Plenty. For example filesystem change events are known to have filesystem and OS dependent behaviours and quirks / bugs.
When a Node module runs a shell command, it's possible that you have a BSD vs a GNU flavour of a tool, or maybe a different version altogether.
The Linux user with which you are running the function might also have different rights which could become an issue when accessing the filesystem in any way.
> VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless.
Maybe, but serverless doesn't answer those questions at all. It just hand waves testing and vendor independent infrastructure.
You shouldn’t be uploading node_modules folder to your deployed lambda so this is an issue of your development environment, not lambda.
> Maybe, but serverless doesn't answer those questions at all.
“Serverless” or lambda/Azure functions, etc, are not a silver bullet that solve every single scenario. Just like docker doesn’t solve every single scenario, not does cloud servers or bare metal. It’s just another tool for us to do our job.
> > No, the lockfile does that.
> your lockfile doesn't care about build steps
Then you're not talking about dependency versioning are you? you're talking about install order. In practice it hasn't been an issue, I should find out how deterministic install order is but I'd only be doing this to win a silly argument rather than anything that has come up in nearly a decade of making serverless apps.
> For example filesystem change events are known to have filesystem and OS dependent behaviours
> When a Node module runs a shell command, it's possible that you have a BSD vs a GNU flavour of a tool
Are you generally proposing it would be common to use an entirely different OS? Or a non-boring extX filesystem?
All your issues seem to come from edge cases. Like if you decide to run FreeB or ReiserFS locally and run a sandbox it it, fine, but know that's going to differ from a Linux / systemd / GNU / extX environment.
> > VMs, docker and having to care about and manage isolation platforms is the reason people started using serverless.
> Maybe, but serverless doesn't answer those questions at all.
Serverless exists precisely to answer the question. I can throw all my MicroVMs in the ocean with no knowledge of dockerfiles, no VM snapshots, no knowledge of cloudinit, no environment knowledge other than 'node 10 on Linux' and get my entire environment back immediately.
I didn't mean build order but install scripts and native module builds.
The first type can create issues when external resources are downloaded (Puppeteer, Ngrok, etc.), which themselves have different versions or that fail to download and where the Node.js module falls back to another solution that behaves slightly differently.
The second type can occur when you have say Alpine Linux that uses MuslC and Amazon Linux uses GCC or when the native module tries to link with a shared library that is supposed to exists but doesn't.
> Are you generally proposing it would be common to use an entirely different OS? Or a non-boring extX filesystem?
I haven't checked but Amazon Linux by default uses XFS on EBS disks so I wouldn't be surprised if Lambda's used the same. So not a boring extX filesystem. ZFS is also relatively common.
> Serverless exists precisely to answer the question.
No, it clearly doesn't because your function will fail in local and succeed in Lambda, or the reverse, exactly due to the issues I mentioned in my various comments here and you will be left debugging.
Debugging which starts by finding exactly the differences between the two environments which would have been solved by a VM or Docker.
...
> I didn't mean build order but install scripts and native module builds.
OK. Then you're still not talking about your dependencies being different. The dependencies are the same, they're just particular modules with very specific behaviour...
> external resources are downloaded (Puppeteer, Ngrok, etc.), which themselves have different versions or that fail to download
That's more a 'heads up when using puppeteer' than a indictment of serverless and a call to add an environment management layer like we did in 2005-2015.
> Linux by default uses XFS on EBS disks so I wouldn't be surprised if Lambda's used the same.
That's worth checking out.
> Debugging which starts by finding exactly the differences between the two environments which would have been solved by a VM or Docker.
I see what you're saying, but planning your whole env around something like a given puppeteer module dynamically downloading Chrome (which is very uncommon behaviour) isn't worth the added complexity.
It kinda is, but not really. You get an event object as param and often only need a few fields from it.
Also, you can run Lambda locally, AWS SAM CLI lets you run them in Docker for debugging purposes.
I stopped using windows for dev machines long time ago, but I'd guess with the Linux subsystem stuff it will catch-up in the next years.
Hopefully WSL2 will solve this.
State could be somewhere else, but if you are not also "pure", you don't have any improvement over a normal service.
This isn't the sense in which servers are usually said to be stateless, however. A Lambda that reads and writes state from a database when invoked is not maintaining state outside of the request-response cycle itself, so it can be said to be stateless.
If your lambda involves persistent storage, your testing might want to wipe or prepopulate the DB with some objects first, but that's not hard and doesn't add complexity, and as mentioned you don't need anything in memory.
I'm probably looking at it wrong I guess. I considered using lambda to unload some CPU intensive calculations / logic and that's it. I figured it would be good for that as long as the latency didn't outweigh any benefits.
* you could argue that 'stateless' should include no long term persistence, but you'd be fighting an uphill battle. Like saying 'serverless' isn't correct because there are servers.
And the “lock-in” in lambda is over exaggerated. If you’re using lambda to respond to AWS events, you’re already locked in. If you are using it for APIs, just use one of the officially supported packages that let you add a few lines of code and deploy your standard C#/Web API, Javascript/Node Express, Python/Flask/Django... app as a lambda.
Testability? Framework adoption? Stability? Industry Skills? Proven Architectures...? Are some of the other question marks I never heard a good answer for.
If you haven’t heard the “right answers” for those questions you haven’t been listening to the right people.
Lambdas are just as easy to test as your standard Controller action in your framework of choice.
The good news; you should be able to accomplish most testing locally, in memory. The bad news; your test code is probably going to be much larger and your going to have to be very aware of the data model representing the interface to the lambda and you're going to have to test the different possible states of that data.
I'll typically write all the core functionality first, test it, then write the lambda_handler function.
Your handler should just accept the JSON value and the lambda context and then convert the JSON to whatever plain old object your code needs to do to process it and call your domain logic.
AWS has samples of what the JSON looks like for the different types of events. You can see the samples by just creating a new lambda in the console, click on test and then see the different types of events.
You can also log the JSON you receive and use that to setup your test harness. I don’t mean an official AAA type of unit test, it can be as simple as having a console app that calls your lambda function and passes in the JSON.
For instance in Python, you can wrapped your test harness in an
if __name__ == "__main__":
block in the same file as your lambda.This is the same method that a lot of people use to test API controllers without using something like Postman.
So you tried it? I don't remember it being hard to set up, at least compared to a DB. Or, you can use the underlying docker images (open source, https://github.com/lambci/lambci) to run your Lambdas in. SAM provides some nice abstractions, e.g. API gateway "emulation", posting JSON to the function(s), or providing an AWS SDK-compatible interface for invoking the functions via e.g. boto3. This way you can run the same integration tests you would run in prod against a local testing version.
When you disclose something, it's a disclosure.
Why do HN folks find this so difficult? It's like putting your shoes on the wrong way around. After you've done it, it's clearly uncomfortable. So don't do it.
But let's be clear, we are talking about commercial products and there is a capital interest in selling these services to all of us devs and engineers.
So while use cases exists and benefits wait to be reaped, as a consultant I strongly feel that we should be MUCH more insistent in pointing out when a product does not make sense instead of jumping onto the hype train.
I am basically surrounded by "LETS TRANSFORM EVERYTHING TO KUBERNETES THIS WEEK!" exclamations, conferences are basically "DID YOU ALREADY HEAR ABOUT KUBERNETES?" and so on ...
It reminds me of Ruby on Rails, a mature and well-developed framework used by global tech firms (correct me if wrong: Airbnb, ~Stack Overflow~, Github) to handle parts of their backend in 2019. But for half a decade now even tiny companies have been screaming around about FancyHTTPTechThisYear (tm) because scale while reporting 1/500th of the traffic of some famous Rails users.
This is not engineering with objectives in mind, it's more akin to the gaming community yelling for a new console.
Stack Overflow is built on ASP.NET: https://stackoverflow.blog/2008/09/21/what-was-stack-overflo...
Software engineering is still a young discipline. Probably half of the people have less than 4 years of experience. They learned from other employees that also had less than 4 years of experience. And that can be repeated for 10 generations of developers.
We are learning, and re-learning the same lessons again and again. Everything seems new and better and then we are surprised when it's not a silver bullet.
Software and cloud vendors do not help. They are the first ones to hype their new language, framework or platform. Technology lock-in is a goal for tech companies offering hardware or services.
> This is not engineering with objectives in mind
Curriculum driven development is a thing. And I cannot blame developers for it when the industry hires you and sets your salary based on it.
We need to be more mature and, as you suggest, think about our technical and business goals when choosing a technology instead of letting providers lock us in their latest tech.
We need to live and breathe a culture that makes even young developers aware that this mistake has been done over and over until a growing collective of developers has recognized the pattern behind this.
After all, in the analogue trades even apprentices are taught many "don'ts" right from day one. Software engineering should not be any different.
Lambda is a good use case for when you have lots of not-often-used APIs. Lambda is a great solution for an API that's called a few times a day. It's also great for when you're first starting out and don't know when or where you'll need to scale.
But once you get to 100rps for an API, it's time to move to a more static setup with autoscaling.
Personally I'd like to see the industry coalesce on "serverless" containers rather than FaaS which are organised around functions being the fundamental blocks. Please just run my Docker container(s) like a magic block box that is always available, scales as necessary and dies when no longer needed.
Say you only have 1k users and don't want to spend too much time on infrastructure, lambda is a perfect fit for it. Your application is taking off and now you have 100k: just click a button and migrate it to a Fargate/ECS. That would be the perfect world.
AFAIK the only framework that supports this kind of mindset is Zappa (https://www.zappa.io). I use it in production but never had to migrate out of AWS Lambda so I'm not sure about the pain points related to it.
Not necessarily. The reason I decided to try this was exactly because I found a tutorial showing you could easily host a bog-standard ASP.NET web app on Lambda with the serverless framework. I had to add a small startup class, and one config file to our existing app and I was up and running.
Sure, they're still FaaS which seems to be the unit of deployment for the serverless movement. For (hidden) managed server container deployment, Fargate is the offered solution I believe.
Not really. I converted a Node/Express API running in lambda using the proxy integration to a Docker/Fargate implementation in less than a couple of hours by following a Fargate tutorial. Most of that time was spent learning enough Docker to do it.
The only difference between the Docker implementation of the API and the lambda implementation was calling a different startup module.
There is nothing magical about any lambda, from the programming standpoint you just add one function that accepts a JSON request and a lambda context.
Converting it to a standalone service (outside of APIs) is usual a matter of wrapping your lambda in something that runs all of the time and routing whatever event you’re using to trigger to a queue.
https://www.troyhunt.com/serverless-to-the-max-doing-big-thi...
To me this is probably the most significant benefit, and one that many folks in this discussion strangely seem to be ignoring.
If you launch a startup and it has some success, it's likely you'll run into scaling problems. This is a big, stressful distraction and a serious threat to your customers' confidence when reliability and uptime suffer. Avoiding all that so you can focus on your product and your business is worth paying a premium for.
Infrastructure costs aren't going to bankrupt you as a startup, but infrastructure that keeps falling over, requires constant fiddling, slows you down, and stresses you out just when you're starting to claw your way to early traction very well might.
This should be a perfect use case for lambda, not "oh you're API is receiving more than 10M req/day? Use Elastic Beanstalk instead with an EC2 instance and ELB". This kind of comment is just to abuse the free tier that AWS provide you with.
The whole idea of serverless is so you don't have to manage infrastructure. OS patching, mucking around with Docker and and port fowarding rules are all removed once you go down the serverless route. Is it worth 8x the cost and 15% reduction in performance? The article argues no. If anything, AWS should price API GW and Serverless more competitively.
It's nice to see though that the Firecracker implementation savings has been passed down to AWS Customers.
If the article saw a 1.5-2 times increase in cost, the debate would be much more interesting.
I'm in the container camp because serverless isn't cheap and you can't have long running processes.
>The whole idea of serverless is so you don't have to manage infrastructure.
...when you are validating your product/market (100 request/day is a success here).
Not everyone on HN is a core dev, tech is getting democratised. So 1 week of time dealing with servers and accounts and infra is a week not asking the right questions.
So, in certain respects it's an enabler. Context always matters.
I agree tech is getting democratised, but when talking about products that allows it, I'm thinking of products like Firebase or the recent darklang, not AWS Lambda. From all the Lambda usage that I've encountered so far, they are still very much in the domain of a "sysadmin".
You only have to setup security groups if you are running lambda's in a VPC, which in the simple case (e.g., not running a cloud extension of an internal corporate network) you probably don't.
https://lumigo.io/blog/to-vpc-or-not-to-vpc-in-aws-lambda/
And, yeah, its a lot simpler than a traditional server setup without that. And even with it.
When there is is no infra to deal with, U/DevX becomes much easier. "Remix this" is pure product joy
This may be a specific criticism of AWS Amplify, but all I've been doing is trading off some arcane knowledge in open source software for arcane knowledge in some vendor-specific system. Most of my time has been spent searching for answers to Amazon-specific issues that haven't been solved in any of their SDKs. Or I wind up walking through their SDK code trying to figure out how to metaprogram my way around it.
Perhaps if I used the web UI to stand everything up, it would have gone a lot smoother. I'll admit I'm a sucker for infrastructure as code. But, as it stands, I can't see how this really saves anyone any time. It's still an incredibly manual process without much in way of help or support the moment you veer off the one path Amazon has decided you should be on.
(To make things more concrete, I still don't have Cognito working with optional TOTP 2FA. The withAuthenticator component apparently only renders the TOTP setup if 2FA is required. I've spent hours trying to work this one out, including spending time on the Amplify Gitter channel.)
> I'll admit I'm a sucker for infrastructure as code.
... which you have to maintain, which probably means Ansible/Chef/Puppet/Salt/etc scripts. Which is fine, it's just a trade-off. I've spent weeks maintaining those kinds of things though, and inevitably when you come back in a year, something's broken.
CloudFormation isn't great, but it usually gets the things provisioned - CF Cognito support sucks though, and haven't been impressed with Cognito at all. Seems too enterprise-y. Like e.g. CodeBuild, which supports 0 open-source use-cases. IDK what some of those teams are doing, but even Azure Pipelines is years ahead.
I think that's a general problem with AWS at the moment. Too many sub-par offerings. Still worth it for S3, DynamoDB, SQS, and Lambda or EC2 (just, the EC2 UX is awful). And those are huge time savers.
I suppose I could look at Aurora instead, although Amplify only supports MySQL Aurora, so no luck with Postgres again. Likewise, the price advantage goes away. And now I'm stuck figuring out a wildly convoluted TCO calculator. I could look at another hosted DB solution, but now I either need to move everything to a new provider or find a way to deal with WAN latency for simple DB operations. Moving means learning yet even more otherwise useless knowledge about a single provider.
I'm sure a lot of this is just growing pains. But as a lone dev just poking at something in his spare time, all these "get running in 30 min. or less" tutorials fall apart the moment you try to do something outside that tutorial. And the number of services available, their weird pricing structures, their implicit deficiencies and explicit limitations, just mean I'm still wasting time not building software :-/
I mainly went with API Gateway since that's the default setup the serverless framework uses.
That's one of the ideas. Serverless shines for burst-y traffic where the traffic timing is unknown. If I had known static high loads I wouldn't use serverless.
You would think that after you tailor the code to the platform and make it stateless, it would cost less for the cloud provider and not more.
This is a very predictable and constant traffic load and not at all suitable for serverless/lambda.
In fact, its the complete opposite.
Many IoT applications are very predictable (anything with regular sensor-readings for example or with repeatable patterns such as people returning from work and turning things on).
So the patterns are (obviously) predictable, but the 'dynamic range' of the load is extremely wide, ranging from stampedes on the cardinal points of the clock face (0/15/30/45 min) to crickets the rest of the time. Some form of elasticity is required unless you are willing to pay for idle servers sized for the episodic peaks.
For example, to avoid the 00/15/30/45 issue you offset yourself randomly over a few minutes within that range to avoid overloading mesh & cellular radio networks.
This has a smoothing effect on the traffic as a whole (as seen from the server) which reduces the "dynamic-range" as you're terming it.
What you propose is one way of doing so. Again, I remind you that an IoT SaaS has multiple tenants and these tenants can't be expected to collectively coordinate. And in certain domains, it is not possible to dictate timing requirements to the client tenant and thus enforce the (SaaS) providers traffic requirements.
See also: traffic jams and people switching on their electric kettles en-masse during intermission of major soccer matches. The first disrupts whole economies and the second has given rise to 'fast pumped storage' at the electrical grid level.
Just because you don't see the accumulated effects does not mean they aren't there.
Plus, receiving 1 million requests in one second, you'll have cold starts for every new container that gets created for you, so achieving consistent low latency would be impossible since you don't control that.
Sure, if you ping your lambdas frequently to keep them from having cold starts, and you’ve worked with Amazon to increase your concurrency limits 100 fold, you might be able to handle 1M requests in a second. But then you’re being charged a lot more than just for those 1M requests.
Why would you need to put a Lambda in a VPC? For access to a database, other EC2 bound resources, or overall network security reasons (such as outbound network monitoring).
Here's the meat, to save you a click:
> AWS says that the one-time setup of the shared network interface can take 90 seconds to complete. But because the network interface gets created when Lambda functions are first created or VPC settings updated, the connectivity-related latency during cold start approaches zero. The same goes for function scaling, which no longer requires new network interfaces for each new execution environment.
Large compute and memory intense stuff isn't suitable for serverless
Do you think you'll have the same opinion in five years time? Or how about three years?
If you look at where AWS makes most of their money, it's selling EC2 hardware.
But no, that's not Amazon, so no one cares either way (pro/contra, cheap/expensive, fast/slow, easy/hard) :|
Running FaaS on AWS without knowing anything about servers and "OS stuff" screams like building the roof without the foundation. And I know that it sounds convenient to run everything as isolated little abstract functions, and ... it works, but it's pricey. But you can do it for cheap, you can deploy Kubless or OpenFaaS (or OpenWhisk or whatever is Apache's equivalent). Naturally AWS exploits this knowledge gap for a lot of their managed services.
Heroku seems to be serverless in the way cloud vendors, including Amazon, have been using it recently; it's not serverless in the sense that Amazon first introduced it, which was a synonym for Lambda. It's pretty similar to GAE, which is definitely within the current usage of “serverless”.
https://aws.amazon.com/lambda/serverless-architectures-learn...
https://aws.amazon.com/fargate/
AWS Fargate is a compute engine for Amazon ECS that allows you to run containers without having to manage servers or clusters. With AWS Fargate, you no longer have to provision, configure, and scale clusters of virtual machines to run containers
And so it seems to miss the obvious middle ground between raw EC2 (or Elastic Beanstalk managed EC2) and full on-demand Lambda, which is modern managed hosting (PaaS, Deis/Dokku/Heroku).
Is this even achievable with Lambda though? Even with Lambda, you still have to configure your "infrastructure", just that instead of ELB and EC2, you now have to manage APIGW and Lambda, and any other "wiring" that you need to put in to do what you needed.
All in all, I can't really say Lambda is really all that "easy" considering options like AWS EB/ECS can be setup relatively easily.
You know those url.py files in a django install? Those are the equivalent of ELB/APIGW thing. You don't manage those, you make them once.
Not sure where you personally draw the line between app development and infra work, but for me "request that comes on this path, calls this function", either via ELB -> backend or APIGW -> lambda, or you know, whatever you use for your django app -> a function defined in a view.py, is app, not infra.
If this were the reason to use Serverless, it doesn't buy you much. Ports? You set that up once and you're done forever. OS patching? You already have to manage patches for your app and its libraries, so OS (really container) patching is just another part of your patching strategy.
The reasons to use Serverless is the same as everything else in the cloud: 1) fast scaling, 2) sometimes it is cheaper depending on X, and finally 3) you don't have to think about where to run it.
This is "you'll always have to do X, so might as well do 10x" argument...
Just because I have to manage patches for 'my app and it's libraries' doesn't mean also handling the OS and everything else that would already be covered by the serverless service is no biggie...
If you're doing a toy demo app, or a website for your local flower shop, yes...
Any non trivial company's systems are not just some random container with a "Nodejs app" inside. They need to keep state, handle caching, redundancy, global CDNs, and lots of other layers.
You either build all those and full ops on top of containers (or whatever), or you get them off the selves and ready to play with a serveless service and their state solution, plug-and-play ready-to-use caching layer, and so on...
You can consider it so theoretically, but for many teams, especially small teams, OS-level systems admin is a different skillset and non-trivial burden that they are glad to pay a premium for, which is why offerings like heroku are successful.
It may work that way for you, but it definitely does not for many many teams.
"You already have to walk to the store, so hiking the appalachian trail is just another part of your walking strategy" -- this is an exageration, the difference between walking to the store and hiking the trail is probably larger than adding OS-level maintenance, but it shows "you already have to do X so doing anything else that I can also call 'X' should be trivial" is a flawed form of analysis, it tells us nothing that you can call all of it "walking" or all of it "patching strategy".
All i had to do was provide a container with a web service that reads a environment variable to listen to a specific port.
You can create a web service in whatever language and framework you want, all you need to do is read a PORT environment variable for incoming messages. The freedom was really awesome, instead of being confined to supported languages and frameworks you get in the functions landscape. Only confinement was that you're pretty much limited to HTTP calls, unless that has changed.
Personally i find the freedom of pick your own language and framework a big positive, and billing is finer on cloud run compared to AWS Fargate(rounded up to next sec) if im not mistaken. I do wish they would make billing even more finer instead of rounding up to the next 100 ms, rounding up to the next 10ms would be nicer when you use non GC or Runtimed languages, like Rust.
There are three use cases for lambda:
1. Event based triggers where something happens externally that you want to process. You can do the same thing with Fargate by either having a continuously running process that reads from a queue or responds to an API request (AWS Event -> SNS -> API).
2. Timed events. You can do this with Fargate directly (https://aws.amazon.com/about-aws/whats-new/2018/08/aws-farga...)
3. APIs. Fargate supports autoscaling.
Fargate can do anything that lambda can do without the limitations - request and response payload size, a 10GB temporary storage drive instead of a half megabyte, and no 15 minute runtime limit. Of course no cold start times since you can have a minimum number of instances always running without Ping hacks.
The point of both is to sacrifice some capabilities in order to remove operational complexity. Lambda just goes farther along that spectrum than Fargate does.
My Docker file that I used to convert my Node/Express API that was running in lambda consisted of...
- setting environment variables (I had to do the same thing in my CF template for lambda)
- retrieving the Node Docker image (one line of code - I had to specify one line of code in my CF template with lambda to specify that is was using the Node runtime)
- copying the files from the directory to the container (as opposed to specifying the locsl path of the source code in the CF Template)
- Specifying the Docker Run command (as opposed to specifying the lambda handler).
- running “Docker build” as opposed to “CloudFormation package”.
How it does it is an implementation detail.
In fact, if you use lambda layers, it’s almost equivalent to building Docker images.
Lambdas relatively fast to spin up and thousands instances can be run at once. But they are time limited running for at most 5 minutes and using 3GB of RAM.
You can do most of the same things with either (after all, they just execute code), but you cannot ignore their very different limitations and that they are conducive to different workloads.
But, also you can have up to 4 vCPUs with Fargate as opposed to one with lambda.
Can anyone explain to me why there's "camps" to this debate?
On one hand you have a docker image with your source code/static binary baked in. The other you have your source code/binary encapsulated in a serverless framework (with an Event handler of sorts as your input).
The container doesn't lock you into a particular ecosystem, but hasn't solved how you deploy and run your container (Heroku, managed k8s, AWS ECS/EKS/Fargate). Serverless on the other hand will give you a runtime for your API and allow you to go live (assuming you've setup an account or whatever).
To migrate something that already works to serverless is a bad idea. Maybe if you're pretty comfortable with serverless frameworks and you like to hack a lot of projects up quickly, sure serverless might be a good fit. But why not instead start off with a container? At least you have the flexibility to run that on whatever you damn want - Even a t2.small instance with a docker daemon running.
I was hosting my personal website on EC2 with an RDS instance and it cost around $20/month. But since the site gets little-to-no traffic and I got sick of EC2 upkeep, I switched S3 & Cloudfront: now my bill is $0.80/month.
Another example: at work we inherited an old EC2 instance that was set up solely to run cron jobs on other servers. Occasionally it would lock up or go down, for whatever reason, creating chaos for the servers that needed specific jobs run frequently.
We switched to AWS Lambda functions for those cron jobs and they've never failed once.
If you're running a games website that gets pinged 24/7, don't use Serverless. But the notion that it's slow and overpriced is misguided if you're applying the wrong use case.
So e.g. if the queries you are making aren't time sensitive or are easily cached switching to s3 can be okay.
I'm fully aware of this but never do it because I'm pretty much incompetent as a sysadmin. I don't want to be in a position where I can break an existing project while trying to configure something new I don't understand yet. Nor do I want to run security updates for a blog I setup 5 years ago.
I'm not 100% thrilled with going serverless because it makes debugging awful, but it has saved me money and it makes most things easier. If I went back to shared hosting, I'd just be paying for the privilege of getting to shoot myself in the foot.
Since the organization only has the one site, there's not enough usage to warrant $5 a month for even a tiny a virtual server.
- Mumble server for my wife's WoW guild (They don't use Discord because some of their peeps are running on really dated hardware that can barely handle WoW, let alone an Electron app running concurrently with WoW)
- Taiga.io task tracker
- Dev server for a web game I'm working on (If I ever decide the game is good enough to publish, obviously I'd run dev and prod servers on separate instances)
- IRC bouncer
- Pi-hole DNS for my phone
Containers? Don't need 'em. And each process requires next to no CPU time, so I always have nearly full CPU credits. RAM is enough unless I'm building the latest version of my IRC bouncer, but hitting the paging file for a few minutes is no big deal.
Source: Seen it happen, am certified.
That's the selling point of AWS in general, but I wouldn't be surprised if (non-tiny) companies are still spending at least as much on AWS/etc. as they would on in-house infra. That is, they're paying for it anyway, and possibly at a premium.
Also AOT compiled languages like C# will run slower on lambda than pre-compiled on an EC2. You shouldn't need benchmarks to work that much out - it's software development 101. However you then went on to compare JIT compiled languages on lambda vs EC2 you'd notice the performance is much closer to each other.
You say tech should be about the results rather than religion and while I do agree there is a lot of cult-like behaviours in IT (and there always has been - emacs/vi, GNOME/KDE/etc, Windows/Linux/MacOS, etc) organisations that really depend on their services genuinely do load test their stuff to hell and back to ensure they are getting the best price, performance, scalability (where required) and resilience too.
That last point is also important when looking at infrastructure. Serverless often (though not always - you need to look at it on a case by case basis) offer you better resilience than rolling your own with EC2 instances. I mean sure, you can span multiple AZs yadda yadda yadda - it's all doable. But again you have to factor in spin up times and spare capacity (eg if you lose an AZ and then have 100% of your traffic over 2/3s of your existing hosts, are they going to have enough spare capacity to survive the spin up time of additional hosts or are you going to lose everything like toppling dominoes?)
Ultimately though, if you predictable traffic and don't have a requirements for redundancy then you could just host it yourself for a fraction of the cost of running stuff in the cloud. The real technical challenge isn't hosting though, it's all the edge cases around what needs to happen when your typical hosting plan isn't enough.
This isn't true. The overhead from the article comes from API Gateway, and has nothing to do with running faster/slow via a Lambda or EC2 instance. The real performance hit in Lambda comes from cold-starts. But if you're willing to accept minor cold starts for not running infrastructure 24/7 then it can be a huge benefit.
You're right that it's not just about the language runtime though. I must admit I'd forgotten about the container spin up et al as well (thankfully the recent ENI fix for VPC lambda's has been released because that was threatening to bite us hard on one project).
[1] https://mikhail.io/serverless/coldstarts/aws/languages/
[2] https://levelup.gitconnected.com/aws-lambda-cold-start-langu...
I haven't tested the new VPC stuff yet to know what the impact is.
https://aws.amazon.com/blogs/compute/announcing-improved-vpc...
They don't contradict each other. They're different graphs demonstrating different workloads. You'd be surprised just how much peoples "real-world usage" can vary from one customer to another.
> I haven't tested the new VPC stuff yet to know what the impact is
Pretty significant for high demand lambda's. Unfortunately I'm not sure how much of what I know is covered by NDA so apologies for not being more specific.
Likely not much of what you know is nda! The article is very in depth.
> Yes, I'm also suspicious about my C# results at this point.
I'm not sure how he got the results he got. So it maybe worth him running the test again.
The JIT cost actually adds a fair bit to the start-up time for C# code. This is why on the webserver there's an option to keep the C# code resident so it's always ready to go. In the past - before this option - it was common have scheduled jobs which pinged your webapp regularly to keep it hot :)
^ https://en.wikipedia.org/wiki/Common_Intermediate_Language
Serverless is all built around fully automated and managed containers, right? So it abstracts the whole runtime environment for webserver code. You use it if you don't have the skills/capability to run your own deployment/scaling pipeline.
The technology is a great 80/20 style leveler; I would highly recommend it to new developers and to people coming from a less technical background (such as design, marketing etc).
But that is what the author did. C# is not 'fully' AOT, it is JITed into MSIL on .net CLR, at least under normal circumstances. Does elastic beanstalk use .net native style binaries somehow?
> But that is what the author did. C# is not 'fully' AOT, it is JITed into MSIL on .net CLR, at least under normal circumstances. Does elastic beanstalk use .net native style binaries somehow?
He's running .NET on EC2, so no cold start times. Elastic Beanstalk is "just" another orchestration layer (and not a good one in my personal opinion but I can see why it might appeal to some people).
It's also worth noting that if you're running the kind of processes that this does become a concern, then lambda is probably the wrong choice anyway. However, as always, the smart thing to do is build then benchmark (as the author did).
Deciding to do a whole infrastructure change to save money without even stopping 5mins to do a quick math on the expected savings seems like a recipe for disaster.
Just switching from m1.small to t2.small would bring their costs down to $55 from $96 for the compute. Just make sure to create a monitor in case you use up all your burst capacity. Other than that a quick reservation would bring that down even further, and all of this with zero re-architecting.
Serverless is super cheap if your workload is not serving requests constantly. It's like renting a car 365 days out of the year instead of purchasing, it only makes sense if you only need it sporadically.
It's obvious from the article that he would have even lived with a slight bump in pricing if he got that.
He didn't decide to do a whole infrastructure change. He just ran an experiment that was about more than just cost.
Spot on! Great analogy.
He never mentions what memory provisioning he uses for his lambda but he does remarks that he has 10M requests a day and that'll cost 10$ a day in lambda costs. Which comes to 0.000001 per request. That puts his service at 512MB memory at 0.000000834 (+ 0.0000002 per request).
If instead he used a 128MB instance (which is reasonable if your request is 3ms, but probably not enough to run .net core) you'll be at ~0.0000004 . Which would put it at roughly half cost. At about ~120$, which is the same as his server setup.
He did mention that he could cut his costs by reducing memory usage.
Seems like there's no reason why it couldn't match up to the server-side cost. I don't think the cost-benefit analysis is as bad as he makes it out to be.
Seems also like there's a very big mismatch in pricing here. If his workload is about 3ms per request but he is charged for 100ms a request, then that's a 33 fold increase in pricing compared to being charge per millisecond. Seems like quite an opportunity for some cloud provider to differentiate itself.
All my congratulations to the "Agile leading company" that provided a 800€/day consultant (probably going to read this) who just followed the tutorial to setup a new project on serverless without any web framework, because you know, "it's the future", and "i like to sleep at night", after they and some of the team had been brainwashed by an AWS event (my initial review concluded with "I admire the courage to start a project without a framework" which was replied with "what is a framework ?"). A waste of hundreds of thousands bucks, but that was not the only mistake : they had 100% code coverage from tests but all tests like all the runtime code was wrapped in huge try...except blocks, I have never seen a team talk as much about "Clean Code" and have such a pile of debt. The Agile company consultant called in recently and said they were sorry, that actually Lambda wasn't good for the use case, and that the consultant was now senior and twice the price.
The company now deploys with Ansible and Docker with compose and practicing what I call eXtreme DevOps (one ephemeral deployment per branch on branchname.ci.example.com), development is back on track, new versions every week... They also trashed the serverless code in favor of a Django Rest Framework app that I coded in 3.5 hours during a night after being there for 2-3 weeks (it was the 6th or 7th week that development had stalled "refactoring" "making lambda happ" "trying to have a local environment"), basically presented it to the team the next day at the retro (where we could see nothing creating value was actually moving forward) and said: make your serverless thing work, or let the frontend people choose what they want to build on : a backend code that became an unusable pile of debt or a backend code that works, presents a self-documented REST API, with an administration interface that they needed, and save the company by creating value instead of "just trying to fit into the proprietary framework that is serverless lambda".
If only AWS was open source, then we could have deployed our own cognito and lambda instances to develop locally, instead of the open source mocks that didn't pass the field test in my opinion: lack of quality, maintenance, documentation were critical factors.
“ First, you don't need API Gateways if you use an AWS SDK. Like this guide shows. And of course you don't need a load balancer, because that's precisely one of the things that serverless takes care of for you :-)
More importantly, you didn't actually compare serverless to your current situation.
With serverless solutions you can typically save cost in terms of server management. Serverless isn't maintenance free, but you should be able to reduce your monthly cost (e.g. lower labor cost).
You're right to research whether it actually works for you, but I think you're not done yet ;-)
Also, perhaps CloudFlare Workers are of interest to you. It's a rather new offer in the market. They charge 0.50 dollar per million requests, with a time limit of 50ms per request. Sounds like this combination would work well for you in comparison to AWS Lambda”
We service ~400 million API calls per month. Half of the calls pull static cached data from a CDN (Cloudfront), and the other half ingests data pushed to it from the SDK (anywhere from 10kb to 400kb per call).
We run of Elastic Beanstalk, which is basically an abstraction later on top of an autoscaling EC2 cluster.
Back when Lambda (and API Gateway) first came out we were excited. It sounded like the perfect way to break free from our “legacy” server architecture.
Until you do the pricing calculations. It wasn’t even close. Managed services like Lambda and API Gateway were prohibitively expensive. (Thank god we didn’t begin development with the services before scaling - we would have never been able to scale at those prices).
Not only price, but performance was horrible. Lambdas take forever to warm up (unacceptable for latency sensitive use cases).
I’ve never really thought of Lambda / API Gateway as anything other than a cool pet toy. Although maybe thats being a little harsh..?
It is, however, a good predictor of the ease of finding people to do the job.
https://engineering.khanacademy.org/posts/original-serverles...
> I only had to add a simple config file, add one dependency and one small startup class to my existing API project.
This is not the way to do it, you have to spend some time on lambda approach.
Usually splitting up your code in to smaller chuncks so you don't end up with long spool up times and you can do things like keeping your lambda warm and caching high impact parts of the code. (sometimes of course you can't cache, I know)
Having gone through this process myself with a very very large code base, it would have not worked to just slap it into serverless.
Going server less can save you money, but not by just taking your existing app and throwing it on Lambda.
A source which supports that. [1]
[1]: https://medium.com/thundra/the-fundamental-problem-solving-n...
More like "seems like there's an easy way for me to try this new thing out, lets give it a shot". It took a couple of hours to switch over to this, and I learnt something in the process. Also, our API is very simple, pretty much all the actions are a single database call, so I don't think in this case we would have gained anything by splitting things up into more pieces. But I can see how it would make sense for a larger, more complicated app.
The point of all these levels of abstraction is it takes less work to manage. Cost and performance is always worse.
It seems obvious that at scale, someone running machines for you could not be cheaper, unless they had access to technology allowing to operate at lower costs or cheaper electricity.
Renting can't be cheaper than at scale under normal circumstances. OP was beyond the breakeven point.
Serverless may change the landscape of development but it's going to take time. The costs need to settle into place while tools and processes need to come together.
Right now it appears it takes longer to build something with serverless, is more difficult to manage for day-2 deployments, and it costs more to operate for things like API servers.
I look forward to serverless doing a lot more in a cost effective manner. It's just going to take time to get there if it ever does.
Even if Serverless performs well for certain sites, I think it is too much of a pain in the ass to set up for local development and deployment. There are better technologies out there that provide easier development setups and easier deployments that can also handle greater loads and compute needs. After using Serverless for the past year, I'm having a really hard time finding where it might make sense. It certainly doesn't belong in our tech stack and I would never reach for it again personally.
To conclude that Serverless is more expensive one would have to compare the calculated TCO of managing EC2 against the line-item cost of Lambda.
The article is also operating at the small change end of the spectrum where cost differences are basically noise. Replacing the TCO of operating a thousand instances with the line-item cost of lambda would be an improved analysis.
This said, of course the optimal choice depends on your use case and profile. In some instances Lambda wins, in others EC2, in others something middle ground like Beanstalk or Fargate.
Hopefully, or else it wouldn't be a total cost of ownership!
> And laying off perfectly capable employees because they are now redundant because serverless has a cost too (financial and emotional)
No doubt. But if your capable employees aren't providing value outside of "keeping the servers humming" there's a very good chance that someone is counting up their fully loaded cost and comparing it to bids from Accenture, Capgemini or Infosys.
I'm fairly ambivalent on the whole push towards serverless but arguments like this feel an awful lot like the pushback against virtualization vs. running bare metal.
But you would usually consider serverless to need less people to manage it. So, even if they all work hard, someone will be not that useful one day or another. Anyway I really don't see all this fuss about doing more with less (or doing more with the same) going serverless if you are already using infra as code and have a 100% automated CD/CI pipeline.
A greenfield project? Totally! But in an established org with good practices? Dunno.
He states that the workload is "saving one event to the database".
And then sees 0.42s for his old solution and 0.48s for the serverless solution.
Even if the server waits for the event being written to disk before it answers the request - that should be a negligible time compared to the roundtrip time he sees.
So the "15%" only hold for his very slow connection. If he repeated the same test on a faster connection, then it could turn out the serverless architecture is not only 15% slower but multiple times slower.
If 0.40s of those "0.42s vs 0.48s" he sees are due to his slow connection, then on a fast line it would be 0.02s vs 0.08s. Then his conclusion would be "4x slower" instead of "15% slower".
This can be done.
https://aws.amazon.com/blogs/networking-and-content-delivery...
They already have a classic ELB in place, so switching to an ALB wouldn't be a significant change in cost or capability (the ALB is often cheaper, though it's hard to tell in this case)
[1] https://github.com/s0enke/cloudformation-templates/blob/mast...
Performance comparison between Fargate and Lambda. AWS Fargate Deep Dive https://www.learnaws.org/2019/09/14/deep-dive-aws-fargate/
If you're really looking to save money, don't touch AWS.
Source: I run the frontend for an interactive "one-page-app" site about 3x larger on a $30 VPS with room to spare.
The backend is:
- $10 VPS with OVH + $10 VPS as a fallback with another provider for my DB. - $5 VPS with OVH to host my central debian package repository.
- About $500 worth of dedicated servers for serving hundreds of TB of files a month (usually below one petabyte though). This is not really relevant to this discussion, because OPs application doesn't have a file serving aspect to it. His application is basically just my frontend and a DB.
- Small files and thumbnails go through cloudflare for another $20 / month. But the main site and most files don't use cloudflare (they'd kick me out if I passed that kind of traffic through them). They serve about 30TB-70TB of traffic a month on my account.
You can replace API Gateway by an ALB, which will be much much cheaper in the article use case.
Lambda is also great when it's triggered by everything but API Gateway. Analysing a file uploaded on S3, reacting to an SNS message or consuming messages in SQS.
Though, your example still does not mean you lost. that 1/100 time is time that developer is not working on something else, that might be worth much more (though if ita 1%, its less likely. but my point is that developer time is more valuable than proportional amou t of his salary).
To me "serverless" implies decentralized networking, with a DHT or some kind of network authority, meaning P2P networking at one point or another.
Am I missing something?
You define a function, you hand that function to a company, and they make sure it runs when certain conditions are met. You then don't have to worry about the server.
AWS API Gateway should be bashed because it truly is shoddy.
Is there anyone who has had success using lambda, or can anyone find a write-up of a success story?
That would help me (and others) understand/believe better I think; if it's a very rare case where lambda ends up being helpful (which is the opinion of this HN thread apparently), a case study write-up of a real world success story would still help us understand when we _aren't_ in that situation.
Many of these APIs are Ajax type status updates, JWT issuers, authentication handlers and other utilities. We use the serverless framework to deploy the code, and it works very nicely for us.
The hate is real in these comments, but for us we are able to move 500 million+/month API requests with minimal code changes, easy scaling up and down, and we were able to learn a little something about serverless architectures and their viability for other possible things in the future.
We used it with great success in our data pipeline. Payload data comes into an S3 bucket, Lambdas will transform that data into a standard format and pipe the result to another S3 bucket. The data then needs to be filtered, post processed, then sent to many different data sources owned by different teams (inside or outside of AWS). Each of these steps are also a Lambda.
The benefits are that up to a thousand concurrent Lambdas can be running, so performance doesn't really degrade with service loads. We can handle requests from dozens of teams all over the world with a team of two developers. We almost never have an unexpected outage. And, most importantly to me, we can add new features very quickly.
For example, when a team builds a model in Python against the data, other teams want access to said model, our team can port it and integrate it into the pipeline in a day or two.
Right now, you can build much much quicker and with a less skilled team using serverless than traditional servers. This is very very important.
Serverless is not static and stuck in time, it's going to change and become cheaper over time as competition increases. Pretty soon everyone except already built products and large businesses will be using serverlesss architecture. The barriers to entry of building a website will be so low, you would be an idiot not to use serverless.
You can’t do that with EC2 nearly as easily or cost effective.
This is where serverless really shines. If you have a fixed static load, you should be using the cheapest available compute instance from EC2 as possible. Reserved instances & test performance in lower cost data centers.
The article doesn't mention this, but I this makes me suspicious that the ELB and Lambda version might be running different versions of .NET Core, which could explain part of the performance difference.
The real gem of Serverless is that if you're using a cloud say for example Microsoft Azure (I'm just more familiar with their offering) you can have events triggered when: a new file/blob is added to your Azure Blob Storage. Another case and I know there's likely a none cloudy way of doing this: when new records are inserted into a database you can run a function. It's cronjobs on crack that will only fail in your code usually, I've never seen a failure of an Azure Function getting triggered, can't speak to other cloud providers offerings.
Yes I agree, it's a lot of marketing hype, and it's even dumber IMHO that they emphasize on the RESTful aspects, and not the useful cronjob like aspects that save you time on figuring out when and how to work on something as it hits your cloud.
8x multiplier on ~1K a month ignores the actual value proposition of serverless. Which is that you're not spending staff hours designing, building, and maintaining scalable infrastructure. If you don't have scalability issues (which at 1K a month, you don't), then it's maybe not the right technology for your use case. If you do, then maybe it's the right technology -- yeah you're paying more but you gotta do the math on what you'd need to build and then discuss tradeoffs with the business. Classic technology cost analysis, this is not new.
I'd also like to point out that Lambda is a serverless option. Fargate is another. I've worked with infrastructures that use ECS + Fargate in cases where a real long-running server is a better spend than millions of lambda invocations. In other words, there's definitely a right-tool-for-the-job consideration to be had here.
Working with new tech is always a learning experience, and it's nice to see concrete numbers about what was learned.
[0]: https://read.iopipe.com/how-far-out-is-aws-fargate-a2409d2f9... [1]: https://read.iopipe.com/how-far-out-is-aws-fargate-part-2-e8...
Lambda meets the requirement of being very low maintenance, reactive, and scalable. I'm not surprised they are charging a premium for it. You don't have to pay multiple $100k sys-admins to keep it running.
What does it cost to host lambdas? Not a lot. So we are good from that PoV.
Will it be commoditized, or will there be crazy markups? Can any random guy set this up? If you really want to check this, try setting one up yourself and see if there are any roadblocks. (I didn't)
tl;dr FaaS lifetimes, lack of cache locality, network rates and isolation make serverless platforms great only for embarassingly parallel, short-lived, I/O-bound tasks that don't require high performance and are only occasionally run. Anything else will be more expensive and slower than just maintaining your own server.
Lambda thus would be perfect for irregular batch jobs or orchestration tasks. Trying to serve an API isn't advisable because of the low-latency requirements.
On a tangential note, AWS Lambda is actually insufficient to serve an API by itself - you also need to use AWS API Gateway to serve incoming requests, create your own route tables and dedicated subnets if you need to allow VPC access, enable Cloudtrail logging and monitoring (very limited), and add IAM rules to allow all of these to interoperate. It is not a turn-key solution by any means.
Add to this the limits on deployment artifacts (50 MB max), and you've basically done all the work needed to launch your own EC2 instance in its own subnet and availability zone, but resulting in something with smaller capacity and worse observability.
Use a server, dammit.
First off, API Gateway might not have been the best choice here; an ALB would probably do the trick for this use case and be cheaper. Debatable if it's fair to include API Gateway in the cost anyway; Lambda is really the "serverless" bit here, but I guess Lambda by itself isn't all that useful if you have no way of invoking it.
Beyond that, they're pushing "slower and more expensive" as a sort of indictment of serverless, that it's worthless. It's a trade off. You're trading price and performance for someone else managing the infrastructure for you. For some people that might be a reasonable trade off, and for others it might not.
Put another way: serverless is more expensive than EC2 -> EC2 is more expensive than a dedicated server -> a dedicated server is more expensive than buying your own and racking it -> etc.
Well, yeah, that's only all true if you only consider the narrow band of cost that you've cherry-picked to be relevant. Obviously buying your own server and racking it has other costs than just the server and the colo space, managing an EC2 instance costs more than just the per-hour rate AWS charges, and so on.
Now, say your company has a mix of low-volume and high-volume tasks--sure, you could comingle your low-volume work use some of the cycles from the high-volume workloads--e.g. add some endpoints to your high-volume application that handles low-volume tasks. But, isn't this trending towards the monolith of yesteryear? Which, is fine, I guess--monoliths aren't pure evil--but with microservices and serverless we've finally been able to apply the tried-and-true design patterns of separation-of-concerns/do-one-thing-well to entire applications (not just to little utilities and libraries).
The point is, there's nothing wrong with embracing the right-tool-for-the-right-job and using a mix of infra strategies is often what's best for an org. You don't have to go all serverless, nor do you have to go all containerized. Let your big-dog 24/7 high-volume applications do their thing in Kubernetes and let your one-off admin tasks spin up for 100ms and die and not bother anyone else.
As in:
> phew! that Serverless stuff turned out to be pure hype, lets dump on it and destroy the hubris of human-kind
But that is just not the case, serverless is a legitimate option. I've seen it work (increasingly well) at various companies.
I will also say, I'm loving the comments here that are trying to boil down the core-principles to follow to achieve serverless success. Great stuff.
- there is a company which provides you with IT administration, which you trust it cpu management, memory management, network management,...
- in most cases (not all, but in most cases), you are developers not system administrators
- you dont have resources in term of hardware budget (in form of multi year plan), network budget, human resources budget, own knowlidge budget,... to set it up as a single man / company operation
Why do you believe, that some company that offers you all those "favors" would make it cheaper than if you wouldnt lack those topics? Even more as there is a huge mountain of knowlidge/time/money/... you dont have?
If I were them, I would spoil everybody to the point where developers wouldnt know what ip is, run operation on my own expenses and then raise up the prices a lot. And I dont think I am much smarter then "them".
To me it is dead simple. Same as all the cloud topics. It is a reasonable trade off, the companies are offering you a "favor" for money in same way that you do it customers.
I looked in to Serverless before, and decided, then and now, it was wholly unnecessary, and I don't know when it will ever become necessary for our org. Azure already had what was necessary - Webjobs. I set up a simple event framework, drop some events into a DB, which get processed asynchronously by some Webjob. It's easy to run on local as it's just a console application, gets pushed to CI and again, is just a console application so it tests the same as other projects, is easily scalable, configurable, and it uses no new technologies.
I always look to see what pre-existing technology or paradigms are already existing that can accomplish the solution, and I found that serverless, in the idea the cloud providers employ it, is usually unnecessary
* if the code you run has heavy initialization phase that could be amortized across multiple runs. Lambda is not a god fit because you'd do initialization for every request. Cold start latency compounds with code initialization to make the problem worse
* if the function workload involves lots of waiting for network I/O (REST APIs, lots of database calls, web crawling). In this case you'll be paying for each lambda function to wait for the network, while if you run the same workload on a single machine it will be cheaper because you can wait for many requests in parallel in different threads without consuming much resources.
Having mechanical sympathy for your workload does help.
https://developer.amazon.com/docs/custom-skills/host-a-custo...
Recently, I created some twitter bots to test out for fun and put them on Lambda with a 12 hr schedule to run from SNS... I was sending 15 tweets a day (from ~15 accounts) and my bill was $35/mth... That is silly
[Anecdote time]: I've seen a product team (of otherwise clueless people) design their whole product around a serverless architecture. In one instance I heard them trying to 'solve' the authentication on each connection problem, where they could have kept connection state and authenticated a user with cookies or the like.
Assuming that some would pay for the slightly slower and more expensive serverless option; is the touted goal of scalability better than n well-optimised instances of a service, where n = 1 for most of the time?
This just reads like a casestudy where it was not the right answer.
I also think people giving others flak for "abusing" the free tiers are missing the point. The free allocation is there to be used. Sure don't go create 5x fake accounts but to me structuring your app to stay within limits is a very reasonable proposition
Its not like the cloud provider is getting screwed - they want as much mindshare to their ecosystem as they can get.And someone learning and deploying their tech gets them quite far on that
For those unfamiliar with Serverless, you can choose how much RAM each function gets. Higher RAM costs more but also gives you a faster CPU. His Lambda functions are, on average, being billed for 500ms (according to one graphic).
If he is running with the base RAM allocation bumps it up he might see significantly faster performance and potentially billed for less.
Tuning lambdas and selecting the most optimal RAM (CPU) allocation is automatic with the help of tools provided by the community.
tl;dr Lambda is billed in 100ms increments and has configurable RAM. Higher RAM Lambda is more expensive but faster CPU. This can reduce your bill.
"...we anticipate that there will soon be a whole generation of developers who have never touched a server and only write business logic."
Honestly, which sounds more like a nightmare than a dream. Knowing what your software runs on and how it works is critical if you want to create REALLY GREAT software.
(reference article: https://www.allthingsdistributed.com/2019/08/modern-applicat...)
1. Highly variable load with lots of concurrency
2. Services that are mostly idle
In the former case, you avoid needing to scale up and down. Lots of excess compute is avoided (and periods of underprovisioning are eliminated while load is increasing). In the latter case, simply not keeping a machine warm during long periods of idle time can save a bit of money, with the ability to pick back up almost instantly.
I would think you'd find it tricky to gain meaningful cost savings with Lambda over another option if your use case doesn't fall into one of those two buckets.
Have a server, but offload resource-intensive tasks to cloud functions. Trying to shove everything into that paradigm wastes unnecessary time.
And not only that, breaking news, serverless with aws is severe vendor lockin.
I also don't find it the least bit surprising that a semi-cold box in a data center can't accept a request, associate it my my account, spin up a worker thread with my stack and service my request in the same time as a box running my stack with a pool of ready worker threads.
It's nice to see someone put some real metrics on it though.
That doesn't mean it's not great at those tasks it's great at.
Example: I have an hourly script I need to run that downloads some files. It's been running as a lambda for 3 years without failure. It's cost me zero dollars (free tier). Even if there were no free tier, it would be less than a dollar total
But if you already are a full-stack engineer, then I think having an entire stack at your hands is way more powerful and flexible.
I think it's matter of time though where just like how Bootstrap paved the way for re-usable components, serverless will soon pave the way for re-usable functions or even services which I think would be a total game changer.
You could also call the above management, and second guessing micromanagement. Even as a senior engineer who has been around a while there is a lot I don't know, and some of that I need to be content to leave to the other engineers to make the mistakes and do the learning not me.
You need to choose based on your load. Also, I would like to see OP slim down his app, so every lambda only has the code it needs to run and also has a permanent DB connection at least (lambda's good practices)
My personal default is to control as much of my infrastructure myself as possible, but at a certain point the edge model can have some huge benefits. Same reason I use a CDN even though it introduces another point of failure.
Serverless is great if you're starting from scratch, and have flexibility with your tech stack. If not, you're better of carving off chunks that make the most sense for lambda (with the upshot that this will make your existing components simplier/faster/etc).
There was a response to a comment I made some time ago that seemed to imply Netflix was doing this (beyond edge delivery) because of AWS costs or funding their direct competitor. A few of us asked for specifics but never got any confirmation that this was happening.
Has anyone been unhappy with cloud?
I have one lambda function in my backend at the moment. It is a trigger from Cognito to create a user in my database. It makes sense here, this lambda will only be run once for each user.
The deal with server less is that while it looks attractive, it, like many AWS products, is great for starting out but often not helpful when running anything for large workloads. Then it turns really expensive.
Are there use cases? Absolutely. Does anyone really understand those use cases? Not really, from what I've seen.
Standalone / Primitive softwares - Uses computing power for running business.
No software - Better humanity. Lets evolve biologically than just 0s and 1s.
But, I love to work in NodeJs for it’s easy scalability , using GraphQL will disrupt the process. What do you think about that ? Thanks
This post is accurate. It's accurate in the new dev experience of someone picking up a framework tool and following the standard walkthroughs/examples that folks like my team and I have been creating the past few years. As far as I'm concerned Einar (the author of the post) did everything I would have done in their position with their experience with this tech. The math Einar did also seems to be accurate given a 1gb RAM and <100ms function duration as well as the API-GW costs, which don't fluctuate based on request duration or Lambda configuration.
Einar got feedback later on that they should have looked at ALB (Application Load Balancer) which does provide a different HTTP request interface back to a Lambda invoke. ALB is the right choice for dead simple proxy requests where you aren't using any of the more advanced features of API-GW. It would lower the cost significantly as well as increase the performance due to a slightly lesser overhead in request path. It would change both the numbers in the blogs title though I can't say it would be 1:1.
We've done a shit job of making this clear and thanks to Einar's post lighting the fire under our butts we'll make work to correct that.
What I am happy about in Einar's post is that they were able to get up and running in almost no time, explore, poke, test, and write this post about their experience very very quickly.
The other thing to highlight here is that I am happy to lose a battle to Elastic Beanstalk, or ECS, or Fargate pretty much any day of the week. All of these exist to significantly reduce the operator burden which compared to running your own container management can be incredibly high. Most companies I've run into in the past 7 years that I've been at AWS have struggled to really measure operator cost but when they compare it to just raw time difference both the technologies I just mentioned and "Serverless" tech like Lambda shift the time spent factor so greatly that cost savings becomes painfully obvious.
Anyway, we're here, we're listening, there's a lot going on behind the scenes (see recent announcement about how VPC cold-start pains will be gone soon).
Thanks, - Chris Munns - AWS - Serverless - https://twitter.com/chrismunns
What you actually buy is abstraction and the need to not manage... Well everything beside code (in simple terms)... It is not an apple to Apple comparison...
At scale/high performance there's a lot of waste with the current pricing model.
Go with t3.small. It's cheaper, faster, and you don't have to watch your CPU credits like you do with a t2.
> We have two because deploying .NET Core in Docker on AWS takes minutes
Remember when mainframes were king, and then people got desktop computers?
The broadband explosion got everyone addicted to always-on Internet. Before this, we had some pretty awesome protocols, like IRC and USENET, etc. which didn't assume your dialup connection was always on.
We need to have the software autonomously link up with one another, with servers just being dumb hubs that store encrypted data. Then they would be commodities that can get paid in crypto. Have you seen MAIDSAFE?
my closest AWS DC is in Frankfurt with HTTP ping below 40ms. if i consider 100ms for actual work, the overhead for serverless is 46%. still impressive, i would have expected >100% there.
I'll start with the initial setup: We can see m1 and t1 instances in there, as well as an m3. These first two are almost certainly running paravirtualized AMIs, and there is a chance the m3 is as well. PV AMIs are going to run slower in most cases to begin with (see: http://www.brendangregg.com/blog/2017-11-29/aws-ec2-virtuali... ) and are running on old instance types that have poor cost/performance ratios compared to modern ones - m1 and t1 are ancient. So, they could likely get even better performance by switching to modern instance types, getting away from old PV style virtualization on Xen, to the more modern Nitro stuff - t3 should be able to replace all of those instances, and you gain access to SR-IOV based drivers for disk and network. Even just switching to HVM instances and going to previous generation t2 would likely be a big win vs. t1.micro and m1.small.
And this brings me to my first point in favor of serverless: With Lambda, you don't have to worry about this. You don't have to keep up on which EC2 instance has the best price/performance ratio, you don't need to bother with migrating AMIs from Xen PV to Xen HVM (or PVHVM) or KVM compatible with NVMe and ENA drivers and all of that. You just manage your code. There's a mountain of other things to worry about when managing your own EC2 instances as well, even beyond this - just general patching, keeping up to date with new kernel versions, worrying about deprecation/EOL of the OS running in your instance, making sure your code works with new versions of glibc and all the other packages when you move to a new major release, etc. You might be able to skate by for a while ignoring all of this, but eventually, it's likely to cause problems. I know what I charge people to help manage all of that sort of stuff, and it's way more than the cost difference between the two setups.
Second: There's no indication if the latency calculation includes cold-start response times. They would likely be enough to drag the average up quite a bit for lambda, and there's lots of documented ways to eliminate them from a production lambda setup. (Though, they'd also likely see a latency decrease moving to t3 for their EC2 based setup as well, as previously mentioned.)
Third: API Gateway might not be the best tool for the job, especially as the author notes they don't make much use of all of the API gateway features. You can just use an ALB instead (or depending on your use case, potentially skip any sort of front end altogether, though it sounds like ALB is probably a better fit for what they're doing) and save on complexity and cost.
All in all, this comparison isn't really showing a setup that is well optimized on either the EC2/Beanstalk or Serverless sides - there's major wins on both the performance and cost sides for both implementations. Without both being optimally setup, it's really hard to draw a good conclusion on which is a better solution from a technical standpoint, and the discussion also completely ignores the cost in managing EC2, even with Beanstalk vs. a serverless setup.
We should accept that most famous CTOs, engineers and other "thought leaders" are not geniuses (also, they're often corrupted by financial interests) and we should not outsource our decisions to them. To innovate, we need to start thinking independently and make our own rational assessments.
Even if you make the wrong decision initially, at least it will be your own decision and you will learn from it. Moreover, when it's your own decision, you will learn from it sooner because you're not going to be getting undue reinforcement from peers.
If you over-idealize thought leaders, you will tend to stick to the popular doctrine instead of listening to your own reasoning.
That goes without saying!
That said, serverless might be 2x slower and 8x more expensive, but all that would be irrelevant if it's e.g. 10x easier, so that e.g. 1 programmer/ops guy can do what it took 10 before.
I don't think anybody uses things like AWS or serveless services because they're faster or cheaper...
Lambda can often actually add to development time because it takes more work to test locally and remote testing adds significant time to the write/test cycle.
Other forms of serverless might save time. However there's always edge cases where they don't and you can lose hours or even days investigating an issue that would be a quick fix if you actually managed the host and/or application.
So the short answer is: not always faster nor easier. It really depends on what you're trying to accomplish.
> I don't think anybody uses things like AWS or serveless services because they're faster or cheaper...
Actually yes they do. But like my previous answer, it depends on what you're trying to accomplish.
This is the problem with serverless (and cloud computing in general). People try to distil it down to "this is better for x" and "that is better for y" but actually the benefits and trade offs are far more granular than that. To the extent that you could have two companies in the same sector releasing competing products and still what architecture makes sense for one company might not for the other.
This is also why nobody ever agrees about these topics online - by the time you've generalised the examples enough to describe them in a forum post, they lose the specificity that makes any accurate design possible.
This has not been my experience at all. Maybe because I have a lot of experience with Lambda, but I find them to be a very fast way to get code written and running.
Granted, Lambda is a great tool for a specific category of task. Namely, when you need to programmatically respond to some event which was triggered by another AWS service and can do so in under 5 minutes. But it's poorly suited for tasks outside of that domain. In that sense, it's like Bash: you don't want to do everything with it, but there are situations where it's the best tool.
The myth: devs are expensive and hardware is cheap
The reality: webdevs are a commodity and AWS is making record profits
The amount you'll pay for a single additional developer, even if it's "just" 70K, is already close (if not double) your whole Clould costs when you start...
If you ever reach the point where you need to handle 100s of millions of customers a month / hiring someone can save you money but perhaps serverless makes sense.
99% of businesses will not reach that level.
Perhaps it's more related to the programmer than the tech stack...
It's also not solely developping, but debugging too. And serverless does make things a lot harder.
There's no real pride in not valuing easy if it's also good.
Cold Fusion had issues that would prevent it from taking over.
~2 years ago I seriously looked into Serverless and very quickly discarded it after seeing a large production deployment of it. I didn't have access to the billing info but the site loaded much slower than a regular site and developer productivity was quite low.
I'm going to stick with good old servers for now. If anyone is curious, a few years ago I did a write up on "Should your next project be built as a Serverless app?" which compares deployment styles in a bit more detail: https://nickjanetakis.com/blog/should-your-next-project-be-b...
Here’s the proof of evil intent behind React, for example:
Or perhaps your idea of "proof" and mine are just vastly different.
i guess we’ll have to wait for the trail of destruction to appear as proof of everything else that has come from Facebook.
If you are managing API gateway or your logging it's a failing on your initial setup and design, not on Serverless.
These don't go away because you went serverless. They are jut now outside your control -- and you will have to wait for somebody else to fix them, usually with no insight into how long it will take leaving you with a very poor messaging you can give your clients as to when things will be back on line.
Simply hand-waving -- this is running on somebody else's server is not going to make the auditors happy.
Enterprise folks don't want to manage servers for a whole host of reasons. But I would say the number #1 reasons is they don't understand the cost, and the #2 is the management team has no clue what they are doing -- and probably running a play book somebody else wrote one time long ago that happened to work exactly once.
It shifts blame, especially if it's in the contract with the provider.
You learn quickly that while AWS is “compliant”, they have enough legal sway to never be on the hook if they aren’t actually compliant.
If you don't adhere to the shared responsibility model and a contractor checks long lived IAM credentials into Github, that breach is definitely attributable to you and you deserve all the "blame" you get.
But if someone figures out how to trick AWS' IAM into issuing credentials that allow PHI to be pulled out of S3, do you think that gets passed along to a customer?
This argument only works until you reach a certain size or security requirements.
At the end of the day, if it's critical to the business it doesn't matter what the paper says, you're still responsible for any damage to the business from a bad vendor.
----
It's just like car manufacturers, nobody cares that Takata was actually responsible for faulty airbags. They simply know their GM vehicle has a dangerous recall.
Rather than thinking about GM’s perspective, think about Takata’s perspective: they screwed up making an airbag, but everything was fine because they outsourced their audit to GM, so indeed it was “GM’s responsibility.” On top of that, GM is still happy to treat them as a partner.
Now imagine that instead of some random third-party supplier, Takata was a division of GM, selling to a different division of GM. The same considerations still apply from the Takata-as-division’s perspective, even though nominally they’re part of a company of that “certain scale.”
Are you joking? Outsourcing compliance critical parts of your environment is the raison d'être of huge swaths of the enterprise IT landscape.
https://aws.amazon.com/compliance/services-in-scope/
Can you imagine how much that page costs in terms of getting the audits, developing the processes & procedures internally to execute on the requirements, etc?
That's not hand-waving away concerns: it's paying a premium to know your infrastructure was managed in scope of whichever compliance regime your business falls under, and to get attestations that prove to your auditors that things are indeed being done properly.
But somehow in marketing materials and blog posts it's instead discussed as if it was a good idea to run everything serverless. Maybe that works great for some edge cases (e.g. reaching PCI compliance), but in general that's not what it's for.
This is becoming unfashionable because it is arcane, but *nix (and Windows I'm sure) really provide some rich functionality and tunability that can let you move fast or reach performance/scale challenges that are very expensive or perhaps out of reach otherwise.
Of course, at one point I was told the same thing about managing your own hardware, and the thought of dealing with the jumpers on the back of the harddrive fills me with the same sense of frustration I'm sure a newer developer would feel when learning their disk is out of inodes for the first time.
Some of us had deploy pipelines into “containers” working like a Lambda in 2003.
Nothing in software is all that novel these days, IMO.
All these features that have been there for a decade plus are just wrapped in a corporate brand now.
Progress.
There is surely truth in this. But also, almost nothing is novel if you watch at the new stuff with the same mindset of the past.
It is wise to realize that most new stuff was already invented, but it is also wise to understand that the surrounding might have changed more than you realize.
(Talking out of my own ass here, just in philosophical terms)
In other words, the Lambda environment, as a development target, is the exact opposite point on the spectrum from an OS like Mosix: rather than trying to make a big cluster of specialized compute resources appear as a standardized local POSIX environment, Lambda instead does away with the POSIX environment entirely and forces your code to interface with the big cluster of specialized compute resource “where it lives.” You can’t just write to disk, because there is no disk. You can’t just talk to other processes on the same machine, because there is no (multiprocessing) machine. In fact, you can’t even cache state in process memory indefinitely, because your process isn’t indefinite like it is in POSIX. Etc.
This is a big benefit for ops folks, insofar as code that’s written as “Lambda code” (but generically, using libraries like Fog) can be easily run in many more modern infrastructure environments (including on-prem ones!) than regular code can, “at scale”, without modification to the code itself. As well, these restrictions (like the one on not caching state in memory) force devs to confront what turn out to be the realities of fault-tolerance, which means far less of an ops headache later on, trying to baby along workloads that rely on never dying.
I would describe Lambda as doing a very similar thing for the server, as what Google’s Native Client (PPAPI) attempted to do for the browser: to create a new, strict ABI for developers to target, that forces code written against that ABI to build its features differently (i.e. more portably, more securely, etc.)
Or, philosophically, “serverless” is just a further extension of Heroku’s “twelve-factor”: it’s an approach that hobbles developers’ available toolkit, for the sake of making the workloads they produce better-behaved and more predictable at runtime.
Not very net new for me, except I’m tethered to AWS.
As usual: ymmv
The thing that was new about Facebook wasn't that it was different than MySpace, but that it was the first social network that managed to scale such that all your friends and relatives ended up using it without your pushing them to do so.
The differentiator of the serverless model is that it's a Schelling-fence development practice, such that you can find real infrastructure components in the wild that are targeted at "serverless deployment" and use them.
Or: there was likely no version of Wordpress developed specifically for your stack; but there is likely some version of Wordpress developed to be deployed on a FaaS cluster. That's what makes "serverless" worth paying attention to. Like Kubernetes, or even Linux: they're the first ubiquitous examples of their archetypes.
Wow, FreeBSD weenies are excited about re-discovering CICS.
See how annoying and wrong you sound?
Citation?
Cite this. I don't believe you. I'm across a pretty broad slice of industry and can only draw on anecdotes from colleagues, but the majority of people with actual hands-on experience are disillusioned and say that the biggest (only?) drive for serverless at this point is top-down organisational pressure created by technically incompetent strategic management that looks to 'thought leaders' and Gartner to tell them what technologies to use.
if serverless works for 80%, then why is that 80% not even remotely reflected in the sentiment here? HN is usually a pretty good gauge for how the wider engineering community feels about a particular technology.
People make more effort to engage if they have strong positive opinions and minor negative opinions. Minor positive opinions, which (you hope) are the average in an audience, get unrepresented if the effort required to give feedback is great. That's why metrics measuring average opinion - like NPS - aim to require as little effort as possible.
Source?
I'll give you that this 80% number seems pretty out there. I don't know how that is measured or what it would be referencing.
If you step back and remove all the commercial software from the argument (something like 50%+ of enterprise workloads, the kind of things you buy from a 3rd party and just run it, like Sharepoint, SAP, or similar) and then look at how many business applications take on a trivial amount of load over time, then the author's post becomes more of an outlier. Few folks have apps that do 100rps realistically. And so for data processing/streaming/batch or web/api workloads serverless actually does work out pretty well. Is this 80%, I am not sure.
There is 100% an inflection point where if your operator cost is low enough(human work+3p tools+process+care and feeding) then the "metal to metal" costs can be comparable. Even the author admits that's leaving something on the floor and so it really comes down to what your organization values most.
I would love for most of our serverless app workloads to be top-down organizationally driven but the reality of it is that it comes often from developers themselves and/or line of business organizations with skin in the game of seeing things move faster in most organizations. This will then typically require buy in from security and ops groups. If these folks you know have the trick to driving top down incompetent strategic management towards serverless I'd buy in on that newsletter.
In terms of HN sentiment and in being a member of this community for almost a decade, I don't know if I'd say it widely represents most of the dev world as it tends to lean way more open-source and less enterprisey. I think there's also a larger number of people that represent IT vendors that would love to see AWS fail here :)
Thanks, - Chris Munns - AWS - Serverless - https://twitter.com/chrismunns
this is me. Thanks, - Chris Munns - AWS - Serverless - https://twitter.com/chrismunns
This is my field of expertise. Serverless in the sense of lambda/functions is not usable for serious analytics pipelines due to the max allowed image size being smaller than the smallest NLP models or even lightweight analytics python distributions. You can't use lambda on the ETL side and you can't use lambda on the query side unless your queries are trivial enough to be piped straight through to the underlying store. And if your workload is trivial, you should just use clickhouse or straight up postgres because it vastly outperforms serverless stacks in cost and performance[1]
For non-trivial pipelines, tools like spark and dask dominate. And it just so happens that both have plugins to provision their own resources through kubernetes instead of messing around with serverless/paas noise.
And PasS products, well.
https://weekly-geekly.github.io/articles/433346/index.html
>One table instead of 90
>Service requests are executed in milliseconds
>The cost has decreased by half
>Easy removal of duplicate events
Please explain.
[1] https://blog.cloudflare.com/http-analytics-for-6m-requests-p...
IaaS is the peak value proposition of cloud vendors. Serverless/PaaS are grossly overpriced products aimed at non-technical audiences and are mostly snake oil. Change my mind.
I don't understand the point you are trying to make.
Edit: The comment I am replying to was originally just 'Please explain' and a link to the article in question, and contained no other context or details.
But we've 100% got customers doing near realtime streaming analytics in complicated pipelines feeding off of things like Kinesis Data Streams. This FINRA example is one datapoint: https://aws.amazon.com/solutions/case-studies/finra-data-val... and this Thompson Reuters one: https://aws.amazon.com/solutions/case-studies/thomson-reuter...
These are nontrivial and business critical workloads.
Thanks, - Chris Munns - AWS - Serverless - https://twitter.com/chrismunns
edit:
-------------------------------------
Missosoup i see you making changes to your comment and it greatly changes the tone/context. i won't adjust my own reply in suit but leave it as it was for your original comments on this.
Not all of it is massive ML models doing a lot of computation, and I've had a lot of success using pandas and numpy in it (and gcp cloud functions).
Serverless has its niche and is a great little tool to smooth the impedance mismatch between data stores.
Thanks, - Chris Munns - AWS - Serverless - https://twitter.com/chrismunns
Turns out I was wrong in this case, but you've explained the situation and everything is hunky dory.
Do you need a hug hesburg?
They also like to join the latest hype more often than not so that should even out the anti-enterprise sentiment. I don't think serverless is over that point yet.
General opinion regarding the topic: I haven't done serverless in any way yet, but if it's similar to "regular"/other cloud services then in my experience it only makes sense of you're so big that building a scalable infrastructure yourself is too expensive (unless you're Facebook or Google). The other use case is if your load is actually fluctuating a lot, to a point where having enough resources available just to handle the peaks at all time is too expensive.
Whenever you can somewhat predict your load, having your own infra is almost always less expensive (at least here in Europe/Germany).
We run on AWS. We log lots of stuff to CloudWatch. CloudWatch allows you to scan for regexps (more or less) and send matching lines to a destination of your choosing. There are about 5-10 such matching events per day that we care about, and when they happen, we want an alert in a Slack channel (or email or text or PagerDuty or...).
Option A:
Stand up a server. Deploy a web service on it that accepts POSTs from CloudWatch and acts on them. Update the box as OS vulnerabilities come along. Pay 24/7 for a server that accepts 10 incoming requests per day.
Option B:
Write a plain Python function that takes the contents of a POST that's already been processed and has no other idea that it's living behind a web server. Drop it in Lambda. Pay for 1000ms of processing time per day.
I wouldn't want to run our entire stack on Lambda and friends, but for certain specific situations it's brilliant and I wouldn't want to give it up.
Or you can upload a single function definition to Lambda and let it do everything else.
Again, I wouldn't use Lambda for everything, but I'll gladly let it handle the stuff I won't want to bother with.
On the other hand, you're probably all set up to do that already. It's no big deal. Introducing a new tech requires training, new deployment procedures, new documentation, etc., and now you probably have a one off component out there. There's a line somewhere, but if I were just adding a webhook for log events I'd throw it on a server I already have.
Complexity and dependency are costs too, and I would argue often more significant costs than the $30/month to run a server.
Lambda is just perfect for such a use case, and the number of processing minutes makes it almost free.
Do most people work in this kind of environment? At my company our management couldn't care less what technologies we use. They want to see reasonable cost and solid uptime, and they ask engineering to deliver that.
What defense mechanisms does your firm have against this? Unless your team leads are golf buddies with the C-suite, it is unlikely that their opinions are going to be taken seriously.
For some reason, in most firms, the c-suite isn't super keen on delegating that power.
So are they indistinguishable or not? Two things can't be a bit indistinguishable.
HN is a pretty good gauge for how a subset of a very startup focused portion of the engineering community feels about technology.
I don't know that the same can be said about it reflecting the opinions of the engineering community on the whole, or if the community is even unified enough for anything to be a representative sample. We're a pretty diverse and opinionated lot.
While we might like to think the opposite, I feel like the majority of people in our profession are like the majority of people anywhere -- going to work to work, and going on FB/twitter/reddit to waste time. I don't know many devs personally who take a big interest in the goings-on of the startup world or open-source world in the large. Do they care about spending some of their free time honing their craft or messing around with a new tool? Absolutely. But few seem like the type who would enjoy engaging in a quasi-philosophical discussion on the strengths of various design paradigms. They spend all their day thinking at work; not many want to continue thinking about the same sort of stuff during their off hours.
....no, no it really isn't. HN is a bit of an echo chamber of people who care about being on the cutting edge. 80% of the rest of the industry goes home and stops thinking about software engineering.
Anyway, here's my two cents: I'm building a web app that has the following components:
* A super simple SPA frontend
* Like...5 API endpoints? that allow people to register their emails to the service
* A mailer that sends out a daily email at the time the user specified
* Like 5 more API endpoints for manipulating some per-user data (profile, preferences, etc).
For this use case, serverless is _perfect_. I'm using the "Serverless framework" which allows you to write the definitions for your functions (think routing rules, a la `routes.py` in a Django app) and CloudFormation directly into a YAML file, then it consumes that, updates a CloudFormation stack with all your bits n' bobs all connected together, and gives me:
* 10 or so lambdas
* Monitoring for all of the above, including pings if something goes catastrophically wrong
* A DynamoDB table to hold all the data
* An API gateway hooked up to all the lambdas
* An S3 bucket with my static assets in it
* A CloudFront distro set up to CDN all my static assets
* A DNS record to hook up CloudFront to my URL
* An SMS (emailer from AWS) queue to ship out emails on an hourly or so basis
This whole stack is declared and managed with like 120 lines of YAML and deploys itself in a minute or two, no extra automation or management required. It costs me like 2 dollars a month right now since I haven't launched the site, but by my back of the napkin calculations, it'll probably cost under 50 bucks a month with my expected traffic. Serverless can be great for this sort of thing, and if I do eventually need to migrate this to VMs running containers or something it won't be _that_ hard (it is, after all, just a pile of TypeScript running on Node).
Now, I'll be honest, it took me a full day of tinkering to figure out how to get static assets in an S3 bucket to publish to CloudFront, but that's because I don't read docs very well, and now I know how to do it for the future. I imagine if I had less hubris and found someone else's tutorial for doing so, I could have had it solved in an hour or so. But lambda does work for a lot of simple use cases.
I imagine that most peoples' shitty wordpress sites from 2006 would benefit a lot from being frozen into static HTML, dumped in an S3 bucket, and put on a CDN. They'd get security patches, OS updates, a global CDN and a bunch of other goodies essentially for free. And the sad reality of our industry is that most of it isn't building fantastically intricate real time distributed systems that both cook your breakfast and brush your teeth, it's maintaining some guy's shitty wordpress site from 2006. Tech isn't sexy outside of the FAANG/HN bubble, and _that's perfectly OK_. Serverless is a great paradigm for those people that don't need anything fancy, it's like the modern LAMP stack IMO. It's a good enough default from a performance and cost perspective for most people who have no business thinking about (or who just really don't want to think too hard about) their infrastructure.
Is it a pile of bespoke garbage that sometimes feels like it's going to topple over? Sure, but so does all the software I work on at my day job. At the end of the day, people should use whatever is right for the task, and serverless is great for low- to medium-volume mostly static sites and apps that have a little bit of data manipulation and a little bit of background processing but nothing that rises to the level of _needing_ to manage your own infrastructure. And that does really cover a significant portion of the industry, even if it's not being adopted by the 80% right now.
I haven't done direct development with lambdas, but I've had to help other devs debug issues (currently in a devops role, but I was mostly a dev for like 15+ years before) and been to a few meetups where I watched people advocate, write and show severless deploys on AWS and Google.
I can see some use cases for it, but I honestly hate losing a lot of the introspection and debugging capabilities with them. I'd personally have to deal with setting up my EC2 instance or K8s/maraton deploy and being able to get to the underlying system if I need to. If things go well with a lambda, then you're all set, but the moment you need to start debugging anything serious, I feel like you're going to be in a world of pain.
In one specific instance, we had a developer who set up a lambda and also setup nginx to route to it because he didn't want to use API Gateway, which he got working, painfully after a lot of work. He should probably put up a blog post on it honestly, as I don't think it's officially supported.
Of course, this is all anecdotal. It's really difficult to measure things like speed and cost objectively with this stuff since everyone's stack and needs and uses are just a little bit different.
Every time something sympathetic to a "hyped" technology is posted it's largely derided for being the "popular, hyped" option. MongoDB absolutely ruined the idea of NoSQL for years because everyone just meme'd about "webscale", without being willing to consider that maybe there's merit to the technology.
Are you seriously doubting that lots of people are moving towards serverless infrastructure?
Even internally at big companies they use "serverless-like" models eg:
* Lots of big companies have an internal FaaS
* Those FaaS are owned and managed by other internal teams, and are treated in much the same way as how you'd treat a 3rd party offering.
Serverless just outsources what those companies are doing already.
I leverage AWS lambdas almost exclusively. Yes I am paying a higher price for worse latency. That's OK. I'll pay money for that, if what I get in return is a managed hardware, os, and runtime environment - problems that generate billions of dollars in revenue for many different companies that are working on making it easier.
Performance is a problem I wish to address aggressively, but it's 3rd on the list. A rough approximation of my priorities might be:
1. Security
2. Complexity (including operational complexity)
3. Performance
Serverless gives me the tools to manage (1) and (2) in a way that has worked extremely well for me personally, and I have not found latency to be a problem for me.
Sadly, the site is down so I can not view the article and comment on things more concretely, however I saw the 8x figure and that's fine. 8x latency is fine for many workloads, but operational overhead isn't fine for mine.
edit: OK, so they just don't support HTTPS. It loads now. 8x cost, 15% performance. Cool.
Cool numbers and article, I appreciate what they're getting at. I don't see how you can walk away with "Serverless is all hype and never an option" from it, but whatever. I will once more point to the "Any time this comes up people bash it because 'hype bad'". The top voted comment currently is someone bashing serverless and contains literally no substance.
> Finally, I'm not trying to bash API Gateway, Lambda or serverless in general here, just showing that for some workloads they are a lot more expensive than boring old EC2 and Elastic Beanstalk
Really makes sense to me! Pick the right technology for your constraints. If serverless offerings like lambda could solve every use case AWS wouldn't need other services.
Serverless is more cost effective for sporadic, bursty things where the cost of an instance sitting idle outweighs the more expensive per-operation costs of serverless. On sustained load, it is almost never cheaper.
Another price to pay is complexity; it's simply more difficult to build, deploy, and monitor serverless systems.
What do you mean by serverless systems?
My experience isn't with AWS, but with Azure, so maybe it isn't an apples-to-apples comparison.
My experience has been that building, deploying and monitoring even moderately complex, interdependent Azure Functions has been less difficult than building the equivalent services to deploy to IaaS or on-prem. FWIW, a big part of that reduced complexity has been offloading infrastructure management to Azure instead our IT guys, but deployment and monitoring has also been practically painless. Are there some complexity "gotchas" that I've missed?
We have a few lambda functions we use that send notifications or perform very specific tasks that are sort of on their own. Running on serverless is perfect for these. These functions are only invoked occasionally (a few times a month). So it is great that the Lambda only boots up the few times it is needed and I don't pay when it sits idle (most of the time). It costs me 10 cents a month instead of say $10 a month with a small EC2 server that might otherwise do the same thing.
Another great use case is bursting or inconsistent traffic/invocation patterns. For example, we have a use case where users can text/SMS in and receive a promotion code. We have this running on serverless because it runs very iradically. A nation-wide commercial will run and ask users to text a code, which triggers hundreds of thousands of Lambda instances within a few seconds. But then it might not run again for a month. On a traditional server platform, we would need a load-balanced server solution with several servers set up for redundancy on a huge influx of traffic, only for those servers to sit idle for who knows how long. You can't shut it off because people might text it at other times too, but you need to have a system that is available for "worst-case scenario" at all times (aka when a national commercial runs). You also don't know exactly when the commercial will run, so you have to always be ready. This setup might costs thousands of dollars a month on a traditional server system, but on Lambda it could cost $100 because we only pay during that huge influx of traffic. What I also like is that you can just trust that the Lamdas will boot up under nearly any traffic load. So you don't have to worry about making sure your server is "ready" or "the right size". If we get 1 hit, then we pay a fraction of a penny. If we get ten million hits then we pay $20 (really - it costs $0.20 /Million). Try to get the same reliability with a $20 server that can do that... spoiler: you can't!.
So I don't look at it as a slower solution that is more expensive. If you are running data analytics or a web server that is always on then this is surely the case. In which place I would say to use a traditional server.
But serverless has a role and it is great for infrequent usage, or for burst traffic. We love that we know we can get nearly infinite traffic and just pay per request without worrying about the infrastructure being ready. It is infinitely scalable, so that we don't need to worry about it.
It is always a pro and a con. I love serverless because it opens up new options for developers and system admins. It is another tool in our toolbelt. It is not the perfect solution for everything. We still run many vanilla EC2 instances for internal CRMs and webservers that run constantly. Or for booting up long computational processes. These are cases where servers are still far better than serverless. If we all learned to understand it as another tool available to us and not a "one or the other" requirement then there is a wonderful world where both tools can coexist.
Note: I use the word "infinite" very loosely in this post. I mean "infinitely scalable" in that it is scalable up to any reasonable amount, beyond what I would ever have to worry about. If the day comes that my servers are getting more traffic than Amazon is able to handle in their data servers then I will be far richer than anyone else here and I will come up with a new solution at that point.
I'm using serverless because it lets me deploy a new API endpoint in a matter of minutes.
There is 0 configuration, and deployment is a single command. I also have my own servers running some of my APIs, the difference how much work I have to do is huge. Serverless on Google Firebase is literally "export a function, now you have a new endpoint!".
There is, quite literally, nothing else to do.
I can move 5x-10x the speed with serverless. There is less (almost 0) infrastructure to maintain.
Yes, it costs more, although the math with Google Firebase Functions is pretty trivial, everything is bundled under 1 service, and sending data around between Google cloud services is free.
Startup, I am adding endpoints on a very regular basis as I build out the app. I probably add a couple a month.
As an example, I am writing one right now. It'll take me ~30 minutes to write+deploy it to test.
My long running and CPU intensive stuff is running on a VM of course.
I'm sure if I had spent several days learning yet another tech stack that I could create a system that let me deploy endpoints to a VM that fast, but I have Better Things To Do(tm).
I could also shove everything behind a single endpoint that has a bunch of options, but that brings in a separate set of problems.
That is where the mental savings comes in.
When I first started, I went from "never wrote an HTTP endpoint" to "wrote a secured HTTP endpoint that connects to my DB" in under 20 minutes.
There is nothing for me to maintain. Setting up a new environment is a single CLI command. No futzing with build scripts. Everything will work and keep working pretty much forever, without me needing to touch it.
The sheer PITA of maintaining multiple environments and teaching build scripts what files to pull in on both my website and mobile app is enough work that I don't want to have to manage yet another set of build scripts. I don't want to spin up yet another set of VMs for each environment and handle replicating secrets across them. I just want a 30 line JS function with 0 state to get called now and then.
(As an example, trying to tell my mobile app's build system that I want a build with production code but pointed at my test environment? Hah! Good luck, the build system is hard coded to only have the concept of dev/prod! I've had enough of that.)
It is frustrations like that Serverless solves.
You could have had the exact same experience with Heroku or many other similar products that have been around for almost a decade and a half--it has nothing to do with serverless.
>The sheer PITA of maintaining multiple environments and teaching build scripts what files to pull in on both my website and mobile app is enough work that I don't want to have to manage yet another set of build scripts. I don't want to spin up yet another set of VMs for each environment and handle replicating secrets across them. I just want a 30 line JS function with 0 state to get called now and then.
It sounds like your architecture is far too complicated if you're managing all this by yourself. You don't need "a set of VMs" per environment if you're a small start up.
>Everything will work and keep working pretty much forever, without me needing to touch it.
If you believe that, you haven't been doing this long enough.
See below with Ander.
I don't have much infra, I don't need infra.
(I do in fact have a couple of VMs because I do have a service that is stateful.)
Not having to think about anything aside from "here is my code, it runs" is nice. Debugging consists of "did I write bad code?"
> It sounds like your architecture is far too complicated if you're managing all this by yourself. You don't need "a set of VMs" per environment if you're a small start up.
For the few things I run on VMs I still have to manage secrets so they can connect to the right DB. I also need to point my apps to the proper environment. The app being dev'd on my phone needs to point to the dev server with the dev DB secrets stored on it. Same goes for the website on localhost, or the website on the development server.
In an ideal world I'd have a script that created new environments on demand and it'd wire up all the build scripts automatically. Every developer would trivially get their own VM and could switch parts of the software (website, app, services) to point to various environments easily.
Why in the bloody world some build systems hard code only 2 options in is beyond me.
> If you believe that, you haven't been doing this long enough.
Sure old runtimes will be deprecated, but let's compare that to:
1. Setting up nginx 2. Setting up letsencrypt 3. Figuring out why my certs auto-renew but nginx doesn't pick them up (Turns out default setup doesn't actually force nginx to pick up the certs!) 4. Learning about various process management tools for JS to keep my services running 5. Setting up alerts for aforementioned daemons. 5b. Probably involves configuring outbound email. 6. Realizing something went horribly wrong in 5b and my inbox is now flooded with alerts. 7. Writing scripts to automate as much of the above as possible. 8. This legit happened, there was a security fix to SCP on Windows that changed how one of the command line parameters worked, breaking my deployment scripts when running them from Windows. 9. Does Google really want to charge me per decryption to store my API keys in their cloud API key storage service? They have to make money, but ouch. 10. Security updates not just for code, but for the VM images.
Like, I get it, if my entire job is back-end development. But I want to write some functions that maybe hit double digit millisecond run times and throw away all their state after.
Firebase says "here, export some functions, we'll give you an HTTPS URL you can call them at."
That is the sum total of my relationship with the tooling. "Run my code." "Sure."
About the only other thing that works that simply is a headphone jack, and those are going away.
If you decide you don't like the service, you're app isn't tied to a proprietary API or architecture.
And you don't need to manage additional VMs for stateful services.
With VMs I have the joy of managing my own keys.
VMs need security patches, updates, and I have to maintain images. I have to think.
It is hard to explain how simple firebase functions are.
There is literally no mental overhead. There is no worrying. There is nothing to maintain or think about, no settings up quotas or managing what machine something is running on or getting user auth setup. No certs that can expire or daemons that can crash.
I can come back 2 months later and count on everything I setup functioning just as I left it.
Some months I wrote a bunch of endpoints, I average 2 a month maybe, but if I am adding a new feature, I can pop out 3 or 4 a day, and then go on my way and work with the front end again.
Getting debugging up and running for them can be interesting (painful and bad), but everything else is just smooth.
I don't want to maintain a bunch of infrastructure, I want my users to ask for some data and get some data back.
If someone is doing non-trivial processing, serverless is probably not for them. But I have a bunch of 20-50 line functions that are all independent of each other and have 0 state.
I realize I'm paying a premium, but it is worth it to have this stuff that just works.
I've seen many projects that do the same without serverless just fine.
It's always been there, unless you had some enterprise-grade stacks. I was able to do this in early 2000s, with PHP and FTP, in a matter of seconds. Just upload that api/new_endpoint.php.
Not like this timing is the big deal, unless you really need to add those API endpoints real quick.
> I can move 5x-10x the speed with serverless.
I'd argue this is only applicable to the initial setup. There is one-time investment in designing your own infrastructure, but you can move fast on anything well-designed.
And it's not like serverless designs are immaculate. Just recently I was listening to Knative introduction talk and my obvious question was "what happens to event processing if servers that power this system fail and processing job just dies without a trace". Turned out, there are no delivery guarantees there yet. My conclusion: "uh, okay, blessed be the brave souls who try to use this in production, but I'll check out in a few years".
> There is less (almost 0) infrastructure to maintain.
It doesn't run on some sci-fi magitech in a quantum vacuum. The servers and networks are still there.
The benefit is that you don't have to design them. Saves you some (possibly, lots of) bootstrapping time when you're about to make your first deployment and ask yourself "how" and "where to".
The liability is that you can't - so when things break your can only wait, or try to re-deploy in hope that it would be scheduled to run on some servers that aren't affected.
Just understand the job and use the right tool for it. This fighting over Serverless and servers is like arguing over whether hammers or screwdrivers are better tools.
HN is a pretty good gauge for how Silicon Valley feels about a particular technology.
I just deployed my first project to PythonAnywhere and I am blown away at how much easier is. I'm going to pass the cost on to clients and that's that. I'm going to be less stressed and can concentrate on what I want to do: build applications.
I don't know if it's "severless" in the same sense as AWS Lambda or whatever, but I can imagine many solo/small shops seeing less ops work as a godsend.
Wow. Sounds like you should not be so certain.
I have also helped engineers at two other companies start making the switch and they are both happy so far... so, in my (obviously limited) first hand experience it works 100% of the time. :)
to put this into perspective we have grown about 10x and our monthly AWS bills have actually lowered. so I have no way of knowing what this workload would cost on EC2, going down that rabbit hole would produce a "savings" number easily into the millions.
A c5d.2xl (8 cores) on aws is about $1875 per year reserved, $500k per year would buy you over 2100 cores; if you only did 100 req/s per core, you'd be able to handle something like 213k req/s average.
Just the alb costs on lambda at this request rate would cost you $10MM annually... 20x the cost of doing it on VMs.
>>> 500e3 / (1875.0/8) * 100 * 60*60*24*365 / 1e6 * 1.51
10158796.8
$500k to spend
$1875 per box / 8 cores per box
100 req/s
60*60*24*365 seconds in a year
1e6 requests per $1.51 of alb spend
Staggering the amount of work you can do with just the amount of money you saved and it seems impossible to me that lambda was able to account for this?We have a unique workload that includes very large bursts of work that need to be worked on ASAP. Spinning up instances on demand is not fast enough, so we had a lot of servers sitting idle. Lambda allows us to execute this same work for much cheaper because we aren't paying for idle CPUs. Lambda also allows us to execute at much higher concurrency than ec2 would, for the cost. 15000 Threads spread across 100's of IPs is easy and cheap on lambda, not so much on EC2.
We went from ~250 customers per cluster(our term for a set of dedicated workload servers) to 2600(haven't pushed it past there yet). The work that was done on those instances is slowly being moved to Lambda and all new work/features are added to Lambda directly. This has effectively allowed us to drop a few clusters and also move to lower resources per instance.
FWIW we don't use ELB for Lambda, they come in via API Gateway, to Lambda/Step Functions directly or into an SQS queue in the case of workloads that have rate limits. For Lambda based workloads it breaks down something like 56% of cost is Lambda, 34% of cost is Step Functions and 9% is API Gateway.
Thanks for the explanation ;)
The idea of running your entire app as serverless sounds like pulling teeth. Thought leaders are free to pat each other on the back on that one... I won't mind being called a luddite.
Those scenarios are probably just as happy with a PHP script on Apache.
[citation desperately needed]
most excuses people give for going serverless is just handing off responsibilities to cloud providers to be able to point and blame if the shit hits the fan.
We switched to AWS serverless about a year ago (API gateway and Lambda) and there have been no real surprises.
I've been able to increase our contractual SLA uptime to customers (and charge for it).
We can kick-off more backend async processes and let the magic of auto-scaling just do it's thing.
If anything, I now have a fear of getting too complacent, as we no longer have a FTE SRE monitoring infra. If something goes seriously wrong, we have to rely on our DevOps AWS console skills to troubleshoot.
But similarly, if the power goes out in California, I have to rely on PGE to fix the issue. Serverless is also a cultural decision to outsource the infra.
How do you know those async requests are completing? What's the percentage of failures? Are you meeting those SLAs? Who's patching the code? How do you test and deploy new versions?
Those are all typically seen as ops tasks, now you're doing them.
Small bit of concern around you saying you're using the console, because good testing generally requires Infrastructure as Code, but I suspect the reality is your operational readiness meets your risk profile. That or you're winging it ya big cowboy. :p
Whether leasing EC2 servers or running serverless Lambdas, the logs all rollup to the same place.
We're gradually using more AWS CLI for things like deployment and policy management. But the console is sufficient for 80% of Serverless DevOps.
And well, I mean if all your guys are just trusted to directly use the CLI or Console, then what you doing for governance and insider threats? Using logs? Blind trust?
As I said, how you're doing things might be fine for your risk profile, but probably not when you could suffer a $1m GDPR penalty, and actually, more than just the money, the ethical issue of fucking over your users matters to some of us.
Not having a go, just saying there are reasons people use terraform, stick it in gitlab, and use CI/CD. You might be too small for that, or you might not have the data, but we aren't doing it for fun :p
I couldn't agree more with this.
I'm trying to get a small library [1] I wrote popular and it's insane how much resistance I get because it doesn't fit the "mainstream religion" of using GraphQL.
People are brainwashed to think API = REST or GraphQL. How many times did I see a RESTful (or even worse GraphQL) for an internal API where RPC would be so so much simpler... (GraphQL is great for third parties but sh*t for internal stuff.)
[1]: https://github.com/reframejs/wildcard-api, a RPC library for Node.js server <-> Browser.
Adding a script that is triggered upon a new message to an SQS queue. The cost in using AWS Lambda is actually in the use of API Gateway, and it doesn't use that, as well.
Now, you're right, I didn't find this from any thought leaders, etc. In fact, I would disregard anyone who couldn't give me a concrete use-case and a cost/benefit overview for a technology. I have met a few executives that more or less were trojan horse leaders for other companies' technologies. Salesforce rings a bell.
The authors use case isn’t well suited to Lambda. In fact, something like ECS would be a much better fit and an improvement over Elastic Beanstalk.
I have used Azure Functions so I will talk to that: There are two options, one is a consumption plan where it is truly 'serverless' and they provide you with server time on demand. The other plan - app service plan - allows you to run these on your existing Azure resources. Another option is to get the runtime on a Docker image and run it on a container outside of Azure completely (but you lose a lot of the monitoring benefits).
Anyway the point is that at least in Azure you can sort of slide the dial to how 'serverless' you want to be - one extreme is the consumption plan, then you can go with the app service plan, then less serverless would be to run AKS (Kubernetes) with autoscaling, then to run VM's with autoscaling, then VM's with manual scaling, then finally perhaps rent a physical server in a data centre, then buy a physical server and stick it in a rack in your office.
The advantage of pure serverless over the less extreme ways is if you have wildly unpredictable loads going from 0 to high to 0 at different times, and don't want to (or can't) set up a reasonable scaling system to suit (or don't want to wait for the scaling to 'kick in').