Guide to Serverless Architecture
simform.com
simform.com
I use AWS extensively, so I can elaborate on AWS's approach to these problems. Deployment is straightforward with CloudFormation, VPCs/security groups/KMS/etc. provide well-documented security features, CloudWatch provides out-of-the-box logging and monitoring. Integration testing is definitely important, but becomes a breeze if your whole stack is modeled in CloudFormation (just deploy a testing stack...). CloudFormation also makes regionalization much easier.
The most painful part has been scaling up the complexity of the system while maintaining fault tolerance. At some point you start running into more transient failures, and recovering from them gracefully can be difficult if you haven't designed your system for them from the beginning. This means everything should be idempotent and retryable--which is surprisingly hard to get right. And there isn't an easy formula to apply here--it requires a clear understanding of the business logic and what "graceful recovery" means for your customers.
Lambda executions occasionally fail and need to retry, occasionally you'll get duplicate SQS messages, eventual consistency can create hard-to-find race conditions, edge cases in your code path can inadvertently create tight loops which can spin out of control and cost you serious $$$, whales can create lots of headaches that affect availability for other customers (hot partitions, throttling by upstream dependencies, etc.). These are the real time-consuming problems with serverless architectures. Most of the "problems" in this article are relatively easy to overcome, and non-coincidentally, are easy to understand and sell solutions for.
- Vendor lock-in: although I try to minimise this and have a clear picture of where the lock-in lies, it is very much present.
- Cold/warm start: this is somewhat annoying. I haven't set up keepalive requests yet, but they feel like an ugly hack, so I'm not sure if I'm going to or if I will just suck it up.
- Security/monitoring: having fewer functions make this less of a worry, but it's still difficult and it's clear that there's not much of an ecosystem and best practices.
- Debugging: this can be really annoying. I can usually avoid this by having a proper local testing setup (see below), but when that doesn't cut it, this gets annoying.
Drawbacks I haven't really experienced:
- Statelessness: this is probably due to not splitting up my code in too many functions, and thus not having that much complex interactions, but this hasn't really been a problem for me.
- Execution limits: haven't run into them yet.
Drawbacks I've managed to contain:
- Local testing: I'm writing Node Lambda's. Since they're just stateless functions, it was relatively easy for me to write a really simple Express server, transform the Express requests into the API Gateway format, and convert the API Gateway response format back into Express format. This works fine for local testing, and reduces the effects of vendor lock-in. That said, I do do a lot of the request parsing in the Lambda function itself, rather than letting API Gateway do this.
- Deployment: this is actually pretty sweet. I'm using TerraForm which, although it's a bit immature and thus cumbersome to set up, has been getting the job done and allows me to easily deploy new copies of the infrastructure for every branch of my code.
- Remote testing: as mentioned above, deploying it for every branch I have allows me to see what it's going to look like in production.
Hmm, perhaps I should turn this into an article sometime...
- a standard function packaging and manifest that each vendor will support
- cheaper + easier tracing, logging, graphs
I feel like this is the most overstated draw back in cloud computing by far. I don't know how most people write their code but me and most people I've ever worked with have a natural tendency to seperate out abstractions to protect you from "likely to change" parts of your application. In fact, if you use Spring boot in Java it already abstracts away a lot of the differences for various cloud scenarios.
Further, in the worst case you have to rewrite some stuff. Before you skip a solution for fear of vendor-lock-in you should attempt a cost analysis: how much will you save on these platform-specific advantages and how does that offset your cost of rewriting when some other solution becomes more desiable? How long will it take to reach break even and how does that time compare to how often software is just rewritten or abandoned in your organization anyway?
Unfortunately I can't edit my post anymore, but I should've noted that I'm overall happy with my serverless setup.
> it was relatively easy for me to write a really simple Express server, transform the Express requests into the API Gateway format, and convert the API Gateway response format back into Express format.
For those interested in this route, there's a plugin[0] for the Serverless Framework that handles this conversion for you. Honestly takes 1 minute to convert an existing Express app to serverless.
Shameless plug but I wrote a blog post walkthrough for anyone interested --> https://serverless.com/blog/serverless-express-rest-api/
Personally I'm staying away from Serverless for now.
[0]: https://nickjanetakis.com/blog/should-your-next-project-be-b...
The post isn't loading so I don't know exactly what it's in reference to (AWS lambda, the framework, the more generic idea?). But, the serverless framework[0] supports all major cloud providers (AWS, Google, & Azure) as well as self-hostable options like OpenWhisk and Kubeless.
The first step in their example is “login to your serverless account”.
The login thing isn't actually necessary (or if it is, that's new)
> - Cold/warm start Should be a temporary annoyance - I'm sure all the providers are already hard at work on fixing this.
> - Security/monitoring Easier to do than regular servers - most FaaS systems have monitoring built in, and collect whatever you write to STDOUT.
> - Debugging: Yeah, can't do much about that.
> - Local testing Besides Up mentioned above, should see more and more shims like https://github.com/RealImage/concave/blob/master/server.js coming up.
> - Deployment Still a mess, but give it time. Heroku like experiences for FaaS should be a matter of time.
I hope that the annoyances are temporary indeed, but that doesn't buy me anything today :) That said, I think for almost all my drawbacks it holds true that they're likely to grow less and less significant, so if serverless is not worth it for you today, it's still worth it to keep an eye on.
Edit: I should perhaps also link the shim that I'm using. It's slightly longer, also specific to my relatively loose configuration of API Gateway, and also simulates S3's client-side file upload functionality. As you can see, it's really not that bad: https://gitlab.com/Flockademic/Flockademic/blob/dev/app.ts
Thanks for sharing though, it's surely useful to some.
No execution limits, since you can deploy and fine tune it to your environment. Can be stateless or stateful depending on configuration.
You can say the same about Object oriented programming or func programming. separation of concerns but on the network. Server functions arent a panecea cause you still have to manage all the other pieces.
An elegant thing would be your entire app is in that single lambda but without state there goes your db. even so, people cant resist taking something and adding and adding more to it.
Interestingly lambda seems to make async IO technologies like nodejs less compelling, because throughput capacity isn’t really relevant anymore.
To make the most of Lambda though you’ll want something that affords small code size (as there’s a 50MB limit), and has fast single execution latency.
There is another way of looking at it: Pay for a percentage of the traffic, irrespective of how fast your application runs or what your budget is. If you reach your monthly budget because of one hacker news trending article, yeah, then no more traffic for you. It can be insane.
If I have autoscaling with an upper bound, I can stay within a fixed budget per month. If I reach my budget, yeah no more new servers, but whatever running keeps running. My business doesn't disappear, I get business continuity. Paying based on % of traffic sounds insane. How will I plan for business continuity if my monthly budget is reached with this kind of model? (without any "minimum xx requests" kind of crap free tier, business continuity means fixed upper bound AND continuing service, not OR)
[0] http://docs.aws.amazon.com/apigateway/latest/developerguide/...
If you don’t have spikiness in your traffic and provisioning for peak capacity is affordable and simple, then serverless is a bit less compelling.
But if you've got time critical applications maybe serverless is not for you.
"Electric lightbulbs: candles with a new name"
https://ebooks.adelaide.edu.au/m/mill/john_stuart/system_of_...
Vendor lock in: The CGI standard is supported by almost every server technology. Plus bigCGI is open source and can be self hosted.
Security/Monitoring: one of my biggest challenges, but separate processes, by their nature, offer separation of resources.
Debugging: CGI scripts are super easy to run locally. I've also configured bigCGI to collect STDERR from apps.
Statelessness: CGI is a stateless standard.
If anyone is interested in this, I'd love extra eyeballs on it, especially for security. https://github.com/bmsauer/bigcgi (Please excuse the bad README, I'll update it next chance I get)
* Manage/deploy servers
* Monitor/maintain/upgrade servers
* Figuring out tools to deploy your app to your server
* Scaling an app globally.
* Coping with outages in a data-centre/availability
* Worry about load-balancing & scaling infrastructureHowever, I'm not sure you can do the same with AWS Lamda Functions. To mean, this means it is like a lock-in.
Excuse me? Let's use lambda for what it was meant to do, not replace your entire stack.
As with most system improvements we're not really talking about removing complexity, just reshuffling it into a new and more pleasing form. Lambda will let you wildly simplify some things, but pain points otherwise accounted for in larger deployments will get moved onto your lambda services to compensate.
Cold starts, for example, aren't generally an issue at the function level in web applications as the application itself is/isn't warmed up. By introducing a Lambda back-end you get to take a stance on that per-function while caring less about it in your webapp front-ends.
The OP is trying to highlight where these new and less obvious challenges arise in the new stack, not pushing Lambda as the end all of app dev.
Basically I handle all the logic in Firebase rules and lambdas. Any destructive action goes through a GCF that updates the Firebase Realtime Database. Actions that happen often (like moving) are updated directly by the client but validated with some overly complicated Firebase rules.
It's very nice not having to worry about servers and scaling, but it can be a really painful development experience.
Let me know if you want to try it out and I'll post the URL.
Now that you've been through it, would you do it again? Or stick to a more traditional architecture?
It's pretty much a collaborative and social experiment where I wanted to explore the concept of using in-browser cryptocurrency mining as a monetization and bot fighting method (something I had a problem with in the original Ludum Dare entry it's based on). That's a different discussion though. (You need to manually start the mining and you can explore the map without doing it. It's only when digging and chatting it's required.)
I'm actually planning on rewriting the whole backend in a more traditional Node.js + WebSockets stack so no, I probably wouldn't do it again for this type of application. However, I will probably use it again for other things.
Such a great idea, I think I should do it as well .. ;). But I won't use Firebase, following your advice ..
Security risk detected: Trojan.Gen.NPE
Whereas, for my digital delivery store, I regularly need to check my logs to see if anyone's doing anything suspecious. For example, a lot of IPs randomly try to visit wp-login.php or /phpmyadmin. Maintaining a production web application is a full time job by itself, if you don't have a team.
Having said that, many people would immediately assume static page builders are generally dumb. That isn't exactly true - You can automate a lot of stuff. For example, my local machine has a custom Jekyll plugin for my store that resizes and optimizes product images before pushing to prod to keep the page load time small. IF I had chosen the Rails/Phoneix route, I'd need to worry about hosting imagemagick or the like somewhere. Or maybe write some code to communicate with an third party API and usually, it's not free.
End of the day, I make sales and that's all that matters. That's when it hit me hard that my customers needn't care nor know what's behind their favorite site.
IF I had chosen the Rails/Phoneix route, I'd need to worry about hosting imagemagick or the like somewhere.
Static sites have no option but to run that stuff ahead of time, but that doesn't mean that dynamic sites can't do the same. Asset Pipelines with precompilation are pretty common - both Rails and Phoenix have one.
But that's the thing - there is no "the host" any more. In the days of yore, if the physical machine hosting your site failed, your site went down. Maybe some rare services had failover and redundancy to some degree, but they were primitive at best compared to AWS (and the other serverless providers, for that matter) today. There was simply no comparison to the ecosystem that exists today. Have you ever had to manage production hosts before? It simply boggles my mind how many people throw out these "the cloud is just somebody else's computer" comparisons - like they've never had to diagnose JVM garbage collection thrashing at 3 in the morning before, or dealt with a server that goes down due to 100% of disk space being consumed by logs, or patch a massive fleet in a matter of hours in response to a CVE, or a power outage, or a hyper-localized network event in a data center, or any of the other million+ super annoying problems that come with managing physical hosts, and to a large degree, VMs/VPSes.
It's true! They don't care in the least, as long as it works and lets them do the thing they came there to do.
Always beats me why people don't warn the readers about the almost impossible nature of logging and monitoring such systems :/
One way is we monitor the Ansible tasks which make this happen. Apart from that, there is no way we can do it.
+ trying to make it happen by using something like a Prometheus+Grafana server would defeat the entire purpose of going serverless
With APIG+Lambda you get CloudWatch integration 'out-of-the-box', and can expand the metrics and logs you send out with a few added lines of config.
You can also toggle X-Ray for a detailed view of your call graph.
If the default implementation isn't good enough, you can define your own CW events, alarms, and log filtering.
I'll grant you that you can't just ssh into a host and tail some logs, but if you're keen you can send your logs to elasticsearch for a better arbitrary search experience.
The other use case I always come across is image scaling.
I'd be interested if anyone would like to share their use cases as entire apps or background processes.
We use Lambda + API GW to manage the glue between our different data/service providers. So for instance we expose a "services" API (API GW) that takes a request, does some business logic (lambda code), calls the relevant provider(s) and returns the aggregate response. That principle can (and probably will) be extended to hosting our own back-end / business logic.
We're trying to get to the point where a dev only needs to write a Swagger file, the lambda code and a bit of configuration, and the rest is taken care of by AWS and our CI framework.
Do you worry about vendor lock in, which is coming up a lot on this comment section?
Well, OK, but how about some examples? How about some advice as to where to draw lines? E.g., if the computations take less time to do on a server than to distribute, then maybe don't?