Serverless Architectures
martinfowler.com
martinfowler.com
* We don't monitor how many instances are running and don't really care. Our "functions" are http endpoints. GAE spins up or down instances to meet the load. Our interface with "instances" is just the size of the bill at the end of the month.
* Async task processing is also just an http endpoint function. We even have a little bit of syntactic sugar so it looks like we're just adding functions to a queue.
* We have no ops or devops staff. We just deploy code that implements functions (http endpoints).
* Persistence is also scaled by Google; there's no database server, just an API to read/write data that scales infinitely (or at least, scales with linearly our bill, not limited by the dataset).
It sounds to me like the article is trying to distinguish between "serverless" and PaaS by describing PaaS done poorly. For the longest time, GAE didn't even expose the # of instances servicing your app. They've exposed a lot more of the underlying constructs since, but you can still ignore them unless you're trying to hyperoptimize your bill.
The answer is that Cloud Functions and AppEngine fulfill different use cases. There's clearly room for both.
There is definitely room for many kinds apps in the PaaS space, including serverless as well as microservices. OP was arguing that existing platforms like GAE have been supporting serverless and can continue to do so. However, the premise of the serverless computing movement is that newer platforms (e.g. OpenWhisk, Google Cloud Functions, AWS Lambda) are better for running serverless applications than traditional PaaSes (e.g. Heroku, Bluemix, OpenShift)
(Lambdas couldn't even be triggered with HTTP calls when AWS Lambda launched.)
Google Cloud Functions is a lightweight, event-based, asynchronous compute solution that allows you to create small, single-purpose functions that respond to cloud events without the need to manage a server or a runtime environment.
Where is the reference to scripting?
"...single-purpose functions that respond to cloud events..."
They are meant to react to events within your infrastructure, rather than being public facing application endpoints.
I generally really like AWS and I use it for a ton of stuff, but I very much welcome competition and innovation in this space.
I'm a massive fan of AWS Lambda, but using it to serve public API quests is just a nightmare. Just because you _can_ do something, doesn't mean you should.
I'm constantly puzzled by why people use Lambda for this - if you don't want to deal with managing traditional services, there's plenty of other options out there (in AWS and otherwise) that are much more suited to the task of serving a website.
We've been discussing Google BigQuery as serverless lately. It's a nice succinct way of saying "fully-managed" + "no-ops" + "no resource deployment" + "seamless scaling" + "separation of storage and compute" + "deployed regionally" + etc. Some customers talk about BigQuery as "Data Center is the Database". So these buzzword help describe what BigQuery is and how it is different from other analytics services and technologies.
In any case, if a buzzword helps get across some meaning, sure, go for it. Folks just get carried away :)
(work on BigQuery)
It's not like web 2.0 which isn't a newer version and also not like serverless when it has servers or cloud which isn't a cloud.
Btw. this isn't about how new it is. Just look at stuff from the 90s. The terms they also didn't stick.
Also those things had a specific well defined meaning.
Like again web 2.0 didn't. User created content, Ajax, etc. all surround that.
Not saying buzzwords can't become meaningful, but they are still buzzwords.
Also they still are wrong when they are something like serverless, which is just plain wrong.
I'm considering using beanstalk for a php app but we have content changes that are very frequent. 134 domains (virtual hosts) and maybe 500 pages (the html) change a day.
How do you update the load balanced set of instances when the instances count changes? tks
(I'm only running on a single box so far, mostly using it because it made deploying to production and all the annoyances that come with that a breeze).
Having said that, this is actually irrelevant when trying to scale out. There's nothing that prevents you from starting your application multiple times each copy on a different machine. As long as your application is stateless (which is a requirement no matter which language you use). I don't think there's any programing language which prevents you from doing that.
[1] http://uwsgi-docs.readthedocs.io/en/latest/WSGIquickstart.ht...
It feels like "intellectual property" all over again —which suggests the rules used for rival goods can be used for ideas, hence ignoring the difference between moving and copying.
We can even deploy that code to IoT devices and embed it inside of desktop and mobile apps.
Serverless frees us from lock-in, but it does not have its own state, so it's easy to become locked into various web services. However, it's also easy enough to rely on your services (microservices) for state, avoiding lockin to other IaaS/SaaS products.
The version of language runtimes is where the most lock-in will occur, and really, that's kinda marginal, too. Language runtimes should be as stable and standard as, say, the Linux kernel's userland interface. If language runtimes can't deal with this, then they'll fail.
Firstly, the author is encouraging the conversion of traditional web pages to single-page web applications, which means that users would now have to download actual software to use the website rather than using the software that they already have and trust: their web browser.
Perhaps most alarming is the acknowledgement of this:
> One of the main benefits of Serverless FaaS applications is transparent production runtime provisioning, and so open source is not currently as relevant in this world as it is for, say, Docker and containers.
This highlights the major issue of SaaSS/SaaS, and FaaS takes it to the extreme:
https://www.gnu.org/philosophy/who-does-that-server-really-s...
Not only does the user not have control (as in the case of SaaS), but in the case of FaaS, the actual author is relinquishing the same control.
These suggestions all "make sense" from an architectural perspective (depending on who you ask, and depending on the software being written). But I plead for others to consider whether this is necessary in their particular instance; it's the default thinking now-a-days: want to write a program? Want a wide audience? Want it to scale? Put it on the Web, make it a service, store it and all user data on someone else's computer, and call it "the cloud".
I expressed my opinions and concerns at LibrePlanet 2016:
https://media.libreplanet.org/u/libreplanet/collection/resto...
Article author here. Don't worry, I'll be laying on some cold, hard, reality later in the series. I definitely don't encourage SPAs universally and I'll be talking extensively about the concerns of handing over responsibility to vendors.
From a technical perspective, your article was very informative, and I did learn from it, so thank you for the information.
Just read your presentation, couldn't agree more with the state of the web.
Ironically, thanks for sharing :)
You mean sort of like every web page that uses javascript?
I agree, that is alarming. Unfortunately, the horse on that has long ago run out of that stable, and most people don't seeem to care.
Some web pages can still function okay without JavaScript enabled, or you can find a way to work around it. Maybe there's even a Greasemonkey script for it.
But it's increasingly more difficult.
> Unfortunately, the horse on that has long ago run out of that stable, and most people don't seeem to care.
As is the case with many things, but the fight doesn't end there.
It annoys me every time I'm submitting something. But evidently not enough to finally edit the bookmarklet.
javascript:window.location="http://news.ycombinator.com/submitlink?u="+encodeURIComponent(document.location)+"&t="+encodeURIComponent(document.title.substring(0,document.title.search(/[-|]/)).trim())VPS (virtual private servers) were available (and largely ignored) for quite a while before 2006, when AWS came along with the catchy word "cloud". This single word changed everything. Same technology all of the sudden became cool, and everybody started using it.
Maybe now it is the turn of PaaS [1] - call it "serverless" and folks finally start seeing all the benefits (true scalability, efficient resource utilization, timely and painless software upgrades, et c.)?
PaaS: Platform (as in Server eg used for API) on demand (as in api.something.com/*)
FaaS: Function (as in eg api endpoint) on demand (as in api.something.com/v1/get_latest_posts)
Of course you can create the 2nd with the approaches of the 1st but the from my understanding the main benefit is that you dont have to - as in: you can just take one endpoint and have it managed by something else.
It's called "shared hosting" and was invented in the 90s.
The distinction between the long running processes and microflows is subtle but important. When started, long running processes wait for an input, execute some code when an input is received, and then continue waiting. In contrast, microflows are started once an input is received, and are terminated by the platform after the code in the microflow finishes executing. One way to describe microflows is to say that they are reactive, in the sense that they react to the incoming data.
[1] http://0pointer.de/blog/projects/socket-activated-containers...
[2] https://developer.atlassian.com/blog/2015/03/docker-systemd-...
Obviously this is tightly locked down with cgroups.
As for having more users than sockets, is that really likely, considering there are 64000 available ports?
AWS Lambda in particular will spin up a new container running their framework, which will then stay alive for a period of time before being terminated.
With socket activation (using xinitd or systemd) and a minimal framework, you could implement much the same on your own VMs.
That's true. I should have pointed out that from the developer's point of view it appears as if the execution is terminated, in the sense that no state is carried across code executions. However, the issue of whether the code is cold booted or not is an implementation detail of a specific serverless platform.
Atlassian did a nice blog post[1] on running Docker containers using socket activation. In principle that makes for a serverless platform but for a one that's difficult to manage. [1] https://developer.atlassian.com/blog/2015/03/docker-systemd-...
If you can't maintain state inside the application, you need to maintain it elsewhere (DB, Redis, memcache), and that elsewhere has a cost associated with accessing it.
Consider session state in a web app. If you keep state in memory and scale horizontally, you need session affinity, so stateless inbound http connections hit the same server instance. That means a session-aware router, delicate and extremely expensive F5s and other "smart" routers. But if the app is stateless and session state is in external storage, then all you need for load balancing is a simple round robin. A cheap router, that's more reliable and requires less administration, for a fraction of the cost.
There are other ways stateful apps are expensive, but that's a big one. I'd rather pay $5k than $100k for a router.
However, it'll be in the middle of the range for startup times, as some agents are sideloaded. Latency is around a second. The initial target is tasks like database migrations and batch jobs, so the ~1s delay isn't an issue.
One of my colleagues in NYC spiked on using one-off tasks for a FaaS and concluded it's not a slam dunk fit.
If I was to shimmy FaaS-style stuff into Cloud Foundry, I think it'd be easier to have standing apps with an array of functions already running, then push the magic into route services. I suspect this is how AWS already does it, given the vast difference in spinup times between Node and JVM requests -- Node makes hot-patching a list of functions easy, the JVM is less hot-patch friendly and really expects you to provide all the JARs at launch time.
Disclaimer: I work for Pivotal, we donate the majority of engineering to Cloud Foundry.
In my opinion, serverless is a kind of a PaaS per definition 1 above. However, in practice serverless is different from PaaS in the definition 2, because Heroku and others have focused on long running services. In contrast, serverless does not replace the need for long running services but works with them. The sweet spot for serverless are short lived, stateless functions, FaaS per your definition. I like to call them microflows to highlight the complementary relationship to the microservices.
Kinda like s3... it involves servers, but you don't have to do anything. Its just a black box you put in or get out data. "Serverless" FaaS is just a place you park some code and it runs.
Yes the name is kinda dumb.
There's merit in serverless, no doubt. There are many things that worry me. Not owning the infrastructure means I don't get the telemetry I'd like, making triage difficult. Direct access to the database is a good idea said nobody ever. And at what point does my own infrastructure become more cost-effective than cloud?
Those concerns don't invalidate the applicability or relevance of serverless though. I think its value as a protyping tool, or to validate a proof of concept is huge.
I am no expert at all, but one idea came to mind: say you have this small festival each year (or once) where you have 1 month (or less) you can buy tickets. Instead of setting up, securing, etc a whole server stack you can simply run te ticket-purchase code on a FaaS and have a database with results. Running your single page app all year on a very simple host, but having this database and compute thing only one month.
Why should this be the case? Why should it cost anything to have an online presence today? Why shouldn't users own their data? In my personal opinion, these types of FAAS (function as a service) and mBAAS (mobile Backend as a service) the article is talking are an enabling technology in this regard.
Maybe there is a secret ingredient to making it work again, but we don't know what that is. Yet.
On AWS, you have options a message queues (SQS), busses (SNS), API gateways, and load balancers (ELB). i.e. many of the existing proven patterns are still around, and you can choose which one is best for the particular circumstance. And that indicates that while they have similarities, they're not exactly the same thing.
I can see cases where a design of "API gateway -> lambda fn -> database" is performant, simple and very cost effective. particularly where the function is very simple and load is low or varies a lot. That won't be everything though.
I don't understand why you think that "Direct access to the database" is always bad. Someone has to access that database directly, on behalf of everyone else who is insulated from it. A lambda fn may not be worse than a web app for that role. But if it is inadvisable, then it can use the same indirect requests that a web app would.
I get all that but I still am unsure of what you mean. A database is useless if no-one can ever access it. So some code somewhere accesses that database, not so? And other code makes request to this code?
What do you consider "client code" and what are you comparing this to? Why do you think that lambda functions cannot be in the second category, and if they cannot be, why can't they use the same gated mechanisms as other "client code", and I would assume that they use restful http to read, and http or message queues to write?
> There's risk of denial of service
in the AWS lambda case, the rate limiting is specifically the role of the api gateway. There's always the risk, and I don't think that lambda increases it.
And yes, of course there's code that accesses the database. The problem is that if it's client code, that code must be trusted by the database.
A better approach is where there's code between the client and the database, also on the server (server estate, at least), that verifies data going into and out of the database.
That isolates client-side changes - malicious, unintentional or otherwise, that might impact the data, because there's a server-side gatekeeper.
So all of a sudden FAAS is more appealing than BAAS, and the FAAS API gateway doesn't just do routing, it does validation too. Guess what, you just created an application/business logic layer on a middle tier, and it's no longer serverless.
"...Why do you think that lambda functions cannot be in the second category..."
They can indeed be in the second category. The author, however, specifically calls out direct database access from the client without lambda functions.
If the database and the lamdba fn are both in AWS, then they're on the same "server estate". How is this any different from the server portion a web app?
No good solution comes to mind, aside from being very explicit. "database client" vs "user client".
If by client you mean "a web browser" then "a better approach is where there's code between the client and the database" makes perfect sense; but in the meaning of "client" where any code that calls the database is a client of the database, it's balderdash.
I honestly don't think that a "trust boundary" has anything to do with it, generally.
Ah sorry, I missed this part:
> 2 Using another example of BaaS, we’ve allowed the client direct access to a subset of our database (for product listings)
"The client" in this sense is a web browser and is thus completely untrustworthy. He seems to be talking about read access, hopefully with an api gateway for auth and rate limiting in front of it. But yes, it is wise to be much more wary of this case.
Direct database access would be like exposing ODBC from the DB directly to a web application. You either have to trust that the application doesn't do a "drop table ..." or hope that the DB has fine-grained enough access controls to prevent that.
Whereas in a SOA design, the API endpoint runs your code that gates access to the DB.
As is pointed out in (1) that can happen already through carelessness. And it does (2)
But it probably isn't what was intended in a newly designed general system. That would assume great stupidity.
I suspect that even the "no code, just access the database" actually has some kind of code and config in front of the db, like odata (3). I'm not saying that odata is great, but it is an example of trivial access to a database. it's gated, mostly read-only and configurable, but it probably doesn't count as "writing a custom server app to gate to a database"
1) https://news.ycombinator.com/item?id=11925043
If you haven't worked with DynamoDB you should check it out, via IAM roles you can assign fine grained access controls per user down to the row level. All of the authorization is done at the IAM level and you don't have to worry about validation or data integrity if the user is read only.
Indeed, 'blameless' would seem more appropriate.
All the best trying to recover a 'serverless' application when one portion fails; lots of late-night phone-calls and picking-through the terms of SLA documents in that future compunded by the fact that most serverless advocates say that you can dispense with operations staff...
So +1
Am I missing something?
And the way services are coupled to the app is about the same. I've used both Heroku and AWS a bit, there really isn't that big a difference here, except that Heroku as a lot more third party services that can be connected.
A real serverless architecture would be federated, like BitTorrent or Bitcoin.
That buzzword means absolutely nothing.
The difference is that traditional PaaS like App Engine and Heroku imply a certain amount of lock-in (although this has gotten better over time), whereas Serverless is a more raw compute service that is no more locked-in than, say, EC2.
> This has a huge impact on application architecture, albeit not a unique one - the ‘Twelve-Factor App’ concept has precisely the same restriction.
While 12 Factor does say that processes should be stateless I've never thought it really meant it. Connection Pools and in-memory caches are pretty typical in 12 Factor (or all non-serverless) apps. And for me that is what makes serverless kinda silly. Some global state is actually pretty useful when you can avoid the overhead of initializing things or going over the network.
The distinction between the long running processes and microflows is subtle but important. When started, long running processes wait for an input, execute some code when an input is received, and then continue waiting. In contrast, microflows are started once an input is received, and are terminated by the platform after the code in the microflow finishes executing. One way to describe microflows is to say that they are reactive, in the sense that they react to the incoming data.
In Django you write a "view", which is just a Python function that is started when an HTTP request is received, and terminates by returning an HTTP response object. Then there are models, but these are just a translation layer between the Database and the view function, they don't keep state either.
I don't see how a Django view isn't a "microflow". It seems to me that the architecture is exactly the same, it's just that you don't get to choose the web framework.
12 factor statelessness I read as meaning more application / domain state rather than infrastructure state btw.
> This is an evolving publication, and I shall be extending it over the coming days and weeks to cover more topics on serverless architecture including some things commonly confused with serverless, and the benefits and drawbacks of this approach.
You can send a tweet to the author: https://twitter.com/mikebroberts
> This is an evolving publication, and I shall be extending it over the coming days and weeks to cover more topics on serverless architecture including some things commonly confused with serverless, and the benefits and drawbacks of this approach.
You can send a tweet to the author: https://twitter.com/mikebroberts
Roughly speaking: A region is a geographic location (e.g. eu-west-1 is in Ireland and eu-central-1 is in Frankfurt, Germany) The "Availability Zones" are separate datacenter buildings in that region connected by fat pipes.
For fault-tolerance, it is common to make sure that all of app's parts run across multiple availability zones in the region. This will happen without much manual config.
Crossing regions is very different - it's not going to happen automatically, they're pretty much isolated from each other. Only a few extremely-highly-available apps (e.g. netflix) have engineered to cross regions. See https://www.infoq.com/presentations/netflix-failure-multiple...
What we do is use a message queue and extreme thread isolation. Are internal framework is sort of analogous to an actor framework and/or Hystrix but is stateless through out. Messages are basically functions that need to be run.
That being said because our whole architecture is message driven and uses a queue with many client implementations we have been experimenting with OCaml Mirage and even some Rust + lightweight container because of the serious limitation of booting up the JVM.
For instance, if I have a machine learning application that has to access a very large trained model, I would like to load that model into memory at application startup. Loading from disk at every "function call" would be too slow. So would making RPC calls to some external "model cache" service.
Does AWS Lambda / similar have some sort of overridable "init"?
That being said, it's important to remember that Lambda doesn't reload your function for every function call. It loads it on the first call and then keeps it active for a few minutes as long as there are still requests coming in, so you'd only have to deal with that initial load once in a while (or almost never if your function is active enough).
Unless you had a way of extracting your model into a very fast database.
...
"(1) is definitely not a concern for the FaaS implementations I’ve seen so far, so we can scrub that one off the list right away."
That point do not make any sense to me. You have to follow the AWS Lambda programming model which is specific to AWS Lambda, so either way you are tied to some libraries and patterns which are vendor-specific.
Is there something wrong that the client browser is connecting directly to the database, so JS -> MySQL direct connection won't expose credentials ?
The client could conceivably connect to something like an ElasticSearch database (JSON API), setup with no auth/read-only permissions and appropriate rate-limiting.
Like, a bit more than FaaS, but less than PaaS?
As an experiment, we built a chat app like this (using Stamplay as the FaaS) and it actually worked.
y/w :D :D