Serverless Computing in 2018
medium.com
medium.com
In general, serverless computing tasks cost 100-1000× more per cpu cycle than a counterpart running on a cloud server.
That means that for tasks that are running regularly, they are a huge loser.
They have unpredictable latency, you lose control of things like logging to disk (you get locked in to cloudwatch or whatever, yet more money and tooling required)
The notion of scale seems promising at first, but I would be very skeptical before I accepted the claim that someone can scale lambda volume from 10 to 10 thousand executions per minute without setting off alarms, dropping traffic, and probably having an engineer at AWS Lamba get paged.
So what are lambdas good for? I'd argue that they make good glue for deployments or batch processes that happen infrequently, don't require predictable performance, and for which having a dedicated server is a little wasteful. I'm skeptical even of those. Remember, a micro server is free tier.
I have had success building prototypes with AWS Lambda, so there's that aspect too.
I'd be interested to hear a counterpoint from someone who uses serverless compute in a large enterprise setting in place of a traditional server stack.
But then you run into all of the problems you've outlined, plus, the tooling process for CI/CD to serverless isn't so great (even with frameworks like Serverless Stack).
That being said, I'm personally excited about the managed Kubernetes offering AWS recently announced. The idea of being able to build my own containers and then just have AWS manage them based on pre-defined configuration I've given them is a lot better than the smoke-and-mirrors Lambda gives you today. For example, how can I tell how many Lambdas of mine are running concurrently? How many are warm or cold? There's lots of visibility lost in Lambda as it is today (not to say they aren't working on improving it).
Another alternative for managing containers on a cluster of machines is AWS ECS. It has been around for a while and integrates really well with other AWS services (e.g. Cloudwatch, ALB). Unlike Kubernetes the scheduler is proprietary, but the agent running on individual instances is open source.
We've been using ECS for a while together with Terraform for managing the whole infrastructure in immutable and declarative way, and it has been great so far.
I think hosted kubernetes was easily the better option for us.
It's only popular because Amazon has poured probably billions of dollars into it and have lobbied their way into many corporations.
Citation needed. With Serverless you have to do zero system administration, patching, maintenance, security hardening, etc.
In the early 1900s, cars required tons of maintenance. You had to hand crank the engine to start it, and if you went on a long trip, you needed to bring along a mechanic, as your car was likely to break down. I know it's tempting for engineers like ourselves to believe we can do system administration better than Amazon, but it's just not true. Do you think automobile owners long for the days when they had to hand crank their engine to start it, and do maintenance just to travel 100 miles? In a decade or two, the idea of doing manual system administration, load balancing, scaling, etc, will be just as archaic.
That is assuming that the system is ready and no further changes need to be made to it. Every time changes are made, deployment of new features do still require administration and security checking. But pretty much the same can be said about non-serverless tech stacks. If you don't make changes to the system then a single person can administer a potentially huge system.
The reality is that systems constantly need to change so serverless doesn't actually add much value and it takes away something very important: Flexibility.
There are many variables that enter into a cost-benefit calculation when thinking about this. But what it really comes down to is Scale. Projects on the low end of scaling will find it cheaper to outsource everything they can. But at higher scale? Think about it like this: Instead of building their own data centers, does it make sense for Amazon or Google to just colo space in other people's data centers to build AWS or GCP? Of course it doesn't.
So, those are the bounds of our "scale scale". At some point it becomes cheaper to move off managed products and onto VMs in the cloud. At some point after that, it becomes cheaper to deploy your own metal. The challenge companies need to solve is determining where they're at, and cost-optimizing parts where they can save money. A great example is Netflix; they're nearly totally on AWS, but they've cost-optimized their CDN significantly to save money on bandwidth by getting out of AWS.
QoS is another consideration. Deploying your own metal is hard, and the quality of the service you provide might degrade. But don't marginalize the other end of that scale; there are QoS downsides to going full managed like with Lambda. Its another variable to consider.
Lambda -> EC2 is the same kind of cost-optimization movement, just further down in scaling. At some point Lambda will become too expensive. Then, you'll probably want to move to something like managed Kubernetes/ECS, which has almost no administrative overhead compared to just EC2 instances but still makes scaling relatively easy.
However, every sysadmin I've met (and I used to be one) would like to think they are managing Walmart's trucking fleet, when the reality is they are more like Joe's local furniture delivery.
Sounds like "serverless" is just a new buzzword for shared hosting, with all the same downsides plus increased vendor lock-in.
1. Enterprise systems are always Rube Goldberg machines. Serverless is perfect for this. Check out the 12 Serverless architectures talk at reinvent.
2. Most enteprise IT shops completely missed containerization. Many are still struggling to adopt the public cloud. The feet dragging and lip service by IT Infra and Security regarding public cloud is real.
3. It’s easier to give someone permission to use lambda than EC2. Thus teams follow the path of least resistance. Infra really fucked up in a lot of shops by allowing lambda but feet dragging on EC2. Teams are skipping over the whole containerization and devops mindset and just going with serverlesss. I’m seeing it happen in my F 500 clients. Teams are designing around lambdas because they know they can get their apps provisioned and deployed easily. Remember bureaucracy is enemy #1 of success so anything that circumvents it will become a well worn path.
Lots of "enterprise" solutions don't need more than a few hundred TPS and can tolerate some of the variability that comes with lambda. The net cost (including ops work) will likely be less in these scenarios, plus they don't have to staff for a devops guy or pay to build that skillset up internally.
Definitely needs better tooling, though I think that'll come along in time.
* Everyone talks of serverless as FaaS or single-file, but most people are not writing single-file contained applications. They are depending on the ecosystem of the language they are using, which means using npm/maven/rubygems/pypi libraries. The dream of merely uploading a single file with a function is just not true, folders have to be hoisted up, packages/dependencies have to be installed and isolated per-function. Buildpacks are there for a good reason.
* Pay-per-request sounds fun until you run into scale. Why build an application that nobody uses and optimize it to remain unused? What happens to business continuity when it receives a huge surge of traffic and runs out of budget? Granted yes, we're talking about idle billing, but anyone running applications at scale - the benefits of having 1-2 always-on, always-budgeted containers completely overwhelms idle billing when it comes to per-request pricing and business continuity.
There are always tradeoffs with any technology. The article comes across as trying to sell unicorns, which is not healthy. It must be more transparent with the bad/ugly sides when trying to cover "serverless in 2018".
I can see that function X costs $Y per invocation and a whole business process comprising of a particular set of functions is going to cost me $Z. With that sort of granularity an architect can give better estimates for new projects or optimising existing. It may even lead to non-technical members of the business building processes by connecting functions together (like AWS Step functions?)
With a marketplace or "Function Store", the code can be switched out for something more efficient or more feature rich. eg. a new image recognition function that improves accuracy at the same time as decreasing execution speed (charged by the author only 1% more per invocation than the previous code).
Its still early days and a lot missing to get to this point (the marketplaces, common APIs, better granularity for things like databases - eg. per row billing) but it can be a compelling vision for Enterprises to get a more accurate idea of the cost of business.
Microservices (and monoliths) should be able to head in the same direction with advanced tracing and operational metrics. Ultimately I think there will be a combination of all three types of architectures rather than a single winning architecture. All of the approaches will need to lead to a better understanding of the costs involved in building and running code. Functions may be slightly easier to achieve that visibility.
To me, this is marketing and second-tier marketing by consultants hoping to sell you their services.
Serverless makes it harder to test locally, harder to move between hosts, often times much slower (cold starts), and forces going to the network to do things like persist to disk, run queues, etc. Of course, all these services are provided by the megacorp in charge of your servers so the lock-in goes deeper.
The only ones that should be considering serverless are companies that need to scale up and down operations at a moment's notice. That is if you're huge or doing a lot of short-burt big data processing. Otherwise, it's all koolaid.
Enjoy the open standards we have. Let's work on improving and evangelizing those instead of megacorp lock-ins.
The people offering serverless are no doubt aware of the difficulties and I wouldn't be surprised if they said they were developing something that would make this easier.
Cold starts are a valid issue though and I feel that the effects of this can be minimized with the use of fast compiled languages such as Go.
Lock-in is also a worry for most, as we progress we need to ensure that we are able to write serverless code in such a manner that they can easily be transported across to other serverless providers.
Warmed-up code will always have an edge. Even if Go code can launch very fast, the same binary retained in main memory and heavily represented in L2 and L3 caches is going to absolutely stomp the same bits being loaded from disk over and over.
> Lock-in is also a worry for most, as we progress we need to ensure that we are able to write serverless code in such a manner that they can easily be transported across to other serverless providers.
This will come down to folks building such platforms. I am working for Pivotal on Project Riff. Oracle released Fn, IBM and others are supporting OpenWhisk. There are really a lot of people working on it.
From the article:
> The most surprising thing I learned from building TimerCheck.io was: If designed well, it is possible for some products built with services like AWS Lambda, API Gateway, and DynamoDB (often categorized as “serverless”) to require zero technical maintenance! The public, free TimerCheck.io microservice API was released in July 2015 and I have never had to do anything to keep it working or to scale it, with one notable and unexpected exception.
> The only time I had to do work on TimerCheck.io was when Amazon declared that AWS Lambda support for Node 0.10 was being phased out. If my code base had been larger, it probably would have been easier to simply upgrade to Node 6.10, but I decided instead to rewrite the function in Python 3.6. It has continued operating smoothly since.
> As part of the rewrite, I also took advantage of one of several good AWS Lambda deployment tools that have arisen to fill what was an important gap when Lambda was launched. I chose Amazon’s open source chalice tool for this project, and was thrilled with how it eliminated all the work of setting up an API Gateway, a formerly daunting task.
> The TimerCheck.io service has had a few short outages over the years, but they were always due to region-wide problems that AWS experienced, and AWS resolved the issue without me having to lift a finger. Even with a lot of work and expense, I could not reproduce that uptime. Letting Amazon operate my service infrastructure more reliably is well worth the cheaper price.
> Speaking of expenses, I’ve been pleasantly surprised by the cost-effectiveness of the AWS services used by TimerCheck.io. I wrote an article that broke down the operating charges for December, 2016 which came out to $11 for 2.1 million API hits. I just checked, and in August 2017 it had grown to 4.8 million API hits for about $20 (“blended price” makes it difficult to calculate exactly).
We use Lambda for ETL type work, Healthchecks, Web Scraping, and infrastructure automation.
> Serverless makes it harder to test locally, harder to move between hosts
Most of our code is written in Java. We have a main method, which calls a static method. The only difference between running locally and running in Lambda is that the Lambda calls the static method while we call the Main method locally.
This also pretty much eliminates lock-in since all of the code behind the static method could without much trouble be moved behind an API endpoint running in a more traditional environment.
> The only ones that should be considering serverless are companies that need to scale up and down operations at a moment's notice. That is if you're huge or doing a lot of short-burt big data processing. Otherwise, it's all koolaid.
Respectfully disagree, as first hand experience has taught me otherwise.
Just like any tool, Lambda is extremely effective when used for the things it excels at.
The number of others using it at the same time has no bearing on price.
You do not have to "notify" anyone at AWS to scale your lambda up to whatever your account invocation limits or spending limit.
Also you can run whatever code you want, native libraries, etc. If you need a cache you can use the disk or connect to some in-memory data store. You can log to wheverer you like, or throw your logs away.
Local development is easy and there are several frameworks that abstract away AWS' wart-like configuration API.
The limitations with lambda mostly have to do with the fact you're basically doing distributed processing with finite resources, and under an ephemeral model. Many libraries aren't written in that way, so you'll need to do some first principles work here and there but I'd argue the resultant architecture, independent of AWS, is worth it. You can share databases, S3 files, whatever.
Generally speaking lambda is good for workloads that aren't always at 100% (basically almost every kind of long-tail application in computing). The cost per CPU cycle is made up when you consider this, and there's nothing to stop people from setting up lamda-style processing on their own clusters, machines, or bare metal if you think you can optimize for utilization better/cheaper than lambda/AWS.
If you're in the 1 or 2% of applications where your system is latency sensitive or always at 100%, you either need to consider other architectures or ways of solving your problem. If that's not possible, lambda might not be for you.
If you're someone that doesn't like new technology and is resistant to change, by all means continue to spend your time provisioning and tending to your flock of instances. You're just going to pass that cost on to your customer or user anyway.
The decision made was to implement another FaaS instead (Project Riff). It resembles Kubeless in a lot of respects. And now I'm assigned to work on it. Funny how things turn out.
Also a quote from unknown source: there's no such thing called cloud, it's someone else's servers.
It's all just Linux. Easily transferable between environments.
You shouldn’t have expected much in the first place.
My money’s on Istio (or something like it) and managed Kubernetes, since they can provide the right degree of insight/monitoring and control (any runtime, as long as you can pack it into a container) and don’t restrict you to any platform.
But I appreciate the feedback, it was somewhat hastily written as I was preparing for the new year celebrations haha but I will take this feedback onboard for my next articles!
I think as a platform to build your core application on, it creates a lot of vendor lock in for very little benefit that couldn’t be achieved using a container scheduler like kubernetes.
I have previously used Lambda with Traveling Ruby and Iron.io.
I think FaaSes have value, but I am still on the fence about their changing "everything".
To start with ...
> When it comes to services such as AWS Lambda, when you expose an endpoint, you don’t necessarily have to worry about massive surges in traffic.
You sorta still do. The latency of a cold start is still higher than a warmed-up chunk of code. Can Node start, interpret code, perform logic and respond in less than 100ms? It certainly can.
Meanwhile, a warmed up JVM can perform the same logic in nanoseconds.
Lambda and most other designs I've seen retain warm copies. Either loaded in advance (I saw an AWS slide deck saying that a large part of their secret sauce is load prediction) or kept around once it has launched.
But you will always have a bimodal distribution, probably quite wide, because FaaSes typically scale down to zero.
Hugging the curve is never free. Either you accept idle capacity (enough to cover a start of a surge while you bring additional capacity online) or you accept lumpy latency.
I suspect FaaSes will shine in situations which are sensitive to billing cost but not latency. Where latency is non-negotiable, you will either see folks sticking to services or some mix of services that are augmented by functions built from the same codebase.
> These Serverless cloud providers constantly monitor and manage the underlying fleet of servers running your code.
PaaSes do this too. It's one of the main selling points.
> Thankfully, every endpoint you set up can use a different language runtime.
This isn't really true. FaaSes currently provide a closed list of supported languages. Implementations vary, even within a single platform.
What's missing here is an open extension point. I have internally argued, and I will be continuing to argue, that we already have one: buildpacks. The concept is well-understood, the tooling and testing is well in hand and we have a great deal of expertise in this area.
In particular, it means we don't need to reinvent a wheel. Identifying that code is meant to be mounted in a FaaSes is no different in kind from detecting that it is intended to be run as a Rails app or a vanilla Rack app.
What's missing from the discussion is what is missing from FaaSes: tools for composing larger systems. Most efforts (eg. IBM's composer[1] tool or Lambda's step functions) look at creating state machines wrapped around functions. I think this is the wrong approach. Composition should be declarative, mediated through state, with no logic outside of the functions. My model here is how Concourse defines tasks. I've been making this argument too and I hope we'll bring to FaaS users the kind of lego-brick composability that Concourse users enjoy.
[0] https://pivotal.io/platform/pivotal-function-service, based on https://projectriff.io/
1. Yeah, there are always going to be situations where low-latency is a must.
2. PaaSes most certainly do, I'm a huge advocate for CloudFoundry usage within my place of work and help onboard people for just this reason. FaaS will simply provide one extra layer of abstraction so that developers won't necessarily have to deal with larger frameworks in situations where it doesn't make sense
3. This is really interesting point of view and I'm inclined to agree with you, there does need to be extensibility that some of these platforms don't currently offer.
P.s. I'm very much looking forward to working with Pivotal's function service offering once it is made available to us!
This is wrong. Most cloud providers, AWS included, will _require_ a heads up for even small surges. There are also caps on capacity; in fact, you need to contact them to lift said caps.
It's solving the market of oversaturated shared-VPS that run Node, Python, or Go? Call when you have PHP.
I have a hard time believing that their other products don't have the same issue when auto-scaling.
API gateway isn't alb or elb and there isn't any recommendation or expectation that customers warm anything up.
I never said "massive surge" but I stand by the idea that 2000 cores and 6 terabytes of memory is massive for most workloads. Of course, that's not the limit and you can request more if you need it.
Lambda isn't for all workloads but AWS (and everyone else) is always changing and improving. You have to check in occasionally to see if old assumptions (e.g. elb) still hold.
Interesting perspective. I quit reading when it said "Python 2.7".
One wonders who's problem they are solving?
AWS Lambda just seemed like too much bureaucratic busywork for no good reason (ugh API Gateway...), last time I tried it out.
I guess it maybe appeals to "enterprise" type developers? Maybe it this way cause Werner Vogels is german and from an academic background? :-) I'm adventurous, because I write this knowing that even referring to national characteristics results in instant down votes here.
Just give me an endpoint where I can respond to HTTP requests and parse the URL path/parameters myself, thank you. That way I can keep it as simple or as complex as is needed, per situation. I just don't want to bother with your insanely over-engineered API of definining custom API requests.
This would be "lambda proxy"with api gateway. Happy to post code samples or whatever.
You can also specify via slightly watered down regex what controller is called, and even do "nothing matched go here"configuration
it's clunky, it needs work, but it's been great for crud or utility stuff