Things I’ve learned using serverless
read.acloud.guru
read.acloud.guru
It turned out to be a complete AWS advertisement as well as hand-waving to bunch of other blog posts without any good explanations. What makes me curious is don't these people actually ever read Hacker News and know what good technical blog posts look like (unless they're just bunch of paid evangelists writing blogs with catchy titles)?
I'm talking about the core business code. Ops is important but replaceable.
Or is that the part you are blind to?
Wut?
There are upsides and downsides to using AWS Lambda, but characterizing it as a walled garden is pretty reasonable. That's not the same as code you can run on any Linux server.
Disclosure: I work on Project Riff at Pivotal.
Aws lambdas have had more refinement done on them. For the time being i wouldn't recommend azure functions unless there's non technical motivations.
I haven't heard much about it other than that it is more friendly open code from the lovely people that brought us Deis and Helm:
https://github.com/Azure/brigade
Hey, I bet you've heard of this, it sounds like Riff is absolutely in the same space :D
I think for most small enterprises today it's not too much to ask that you have a Kubernetes cluster with autoscaling provisioned somewhere. I think in 2018 you're not serious if you don't have at least that (or something comparable, although I've heard "the war is over" and agree that people should just get comfortable already with the idea of K8S if they haven't yet)
There are enough managed offerings today that don't charge anything for masters, where you can simply push a button and get a cluster that is properly configured, and push another button to tear it down when you're done, or call an API and get the same effect.
I know that's not really "serverless" now, and it's all about the cost of running computers in the cloud on a 24/7 basis, so tell me if you've heard this one before...
I've never succeeded in standing up a Kubernetes cluster with ASG for workers that will scale all the way down to zero when demand for worker nodes evaporates for a long enough period of time (10-30 mins?). Admittedly I've never spent that much time trying at it either... I am privileged to have some real physical computers plugged into the wall that I don't have to turn off, so I guess I just don't have to think that way.
There's just not any technical reason that won't work though, is there? You'll need the master(s) to hang around, so it's possible to notice Pending pods and scale back up when the demand returns, right?
(So why am I not seeing this capability advertised or demo from any managed Kubernetes provider offerings, is it really just simple economic answer that given the pricing model of no-cost masters, they don't make any money off you during a period of time that you aren't running any worker nodes?)
I have, and I admire a lot of the work Deis folk have been doing at Microsoft. I have different opinions about the future looks, but I could be wrong. And I'm not the only member of the riff team.
In terms of "scale to zero" for workers, I think your "two whys" need is containers on-demand, not workers on-demand. That need is going to be met by the various virtual kubelet efforts underway. Azure have been out front on this, actually, with AWS Fargate coming hot on their heels. I expect that as GKE matures it will hit this too.
As we move towards "five whys", it turns out that we are essentially re-treading the path that Cloud Foundry got to years ago (and Heroku before that): focus on making it easy to run code.
Containers are in themselves an almost-irrelevant implementation detail 99% of devs should never have to care about, just in the same way that most of us don't think about mallocs any more.
I call this the Onsi Haiku Test, after the `cf push` haiku that Onsi Fakhouri gave at a conference a few years back:
Here is my source code.
Run it on the cloud for me.
I do not care how.
I'd really like to get you in the room with a couple of architects and technology leadership in my office. (No seriously, maybe zoom room.)
I'm on the kubernetes train, but they are mostly still hoping on Fargate, having never made this leap, and I have this feeling that I never would have got into the k8s world without the kind of help I got from Deis.
> Containers are in themselves an almost-irrelevant implementation detail 99% of devs should never have to care about
Couldn't agree more. Deis made this easy for me before it was on Kubernetes (CoreOS and Fleet), and when I was finally convinced to leave that stack behind, Deis made it easy for me again to do the same on Kubernetes. I'm the biggest fan of Deis anywhere.
(I've felt the loss of the Deis Workflow maintainers so badly that I'm personally working on the team to fork Deis! But the bus factor is way too high for my place of work, which is a university; they want something they can understand and that they can support or pay a vendor to support if I am not around anymore. That won't stop me, but it also means I need to keep an ear to the ground for something we can use to start doing CI/CD here.)
The technical leaders in my place of work, have already made the leap to AWS, but are just testing the waters of eg. spot market and serverless (lambda) to try to get the cost and reliability benefits to start to materialize, and they would really like to skip containers altogether and start building everything for Lambda. I know enough to say "whoa there Icarus that's no way to reach Lift-and-Shift" and pretty sure from my experience you should start lower (but still with some higher abstraction than plain old Docker containers, and also not Compose or Fargate.)
So I'm in a pickle because Deis is no longer offering support for end users, otherwise that's probably what I'd still be recommending.
I've been looking at possible replacements like Cloud Foundry (and Convox, and Empire) but your haiku hits me right in the feels and is the really important message I need to deliver. I am developing an application right now and I need the kind of devops machinery and support that is appropriate for that kind of effort in 2018
(and I definitely don't want to be embroiled in exploratory project to implement containers for the whole organization some time in the next 5 years, at least not before we can get something out the door for our customers across campus...)
I just don't think we do enough software development to justify spending on something like PCF but I'm not the one who would need to be convinced, either!
You can run OSS Cloud Foundry (now called Cloud Foundry Application Runtime or CFAR) using BOSH and cf-deployment. You can also run Kubernetes with the same operator tools if you use CF Container Runtime (CFCR), for people who need that capability.
SUSE sponsor an OSS GUI called Stratos.
For CI/CD, I am alllll about Concourse. Automation-as-a-Service is a secret gamechanger.
My work email is in my profile if you'd like me to hop on a call with anyone.
When using percentages you always have to ask 70% of what?
What happens when/if Amazon changes their offering to something that makes your system incompatible overnight? What about your keys getting filched and you inadvertently run 100 GPU bitcoin clusters?
How much would you expend in doing an emergency mass migration somewhere else? Would your company even survive?
People who choose to use Amazon exclusive APIs will get bit. It's not an if, but when. I'm not saying "dont buy ec2 instances or s3 storage instances"... Those in the end are just VMs and storage that you can purchase elsewhere. But whom else runs "Lambda"? What is your migration plan if they they cancel your service/quit offering/not offer it?
They wont, or at least historically have not. There's no guarantees however, but this seems like a low risk.
More likely is that AWS will either raise their prices (or are undercut by a competitor) such that it makes financial sense to migrate to a new platform.
I remember something very similar happening to a FireBase customer, in which surprise billing and something occurred that caused them to go from $10/mo to $1600/mo. That's the class of "oh shit" I'm talking about.
We didn't get charged for the work, though we did have to talk to Amazon rep to alert them of what had happened.
It's good architectural design (these days) not to marry yourself to your underlying platform. As a core system design, Lambda is worrisome for me for that vendor lock-in
I’m betting on Amazon being in business at least as long as IBM. The benefits far outweigh the costs of having to port my code in 100+ years. If the machines aren’t sentient by then...
RDBMS systems could handle billions of complex queries per day in the 1990s (ie last century) and ship with an entire language designed to allow you to safely, incrementally evolve your data model.
MySQL and ORMs are not the limits of that universe.
I broadly think so too: https://news.ycombinator.com/item?id=5262556
Though the shifting sands of the economics of compute, disk and network tend to favour this or that approach as time goes on. So while FaaSes are just CGI, they aren't just CGI; but we can at least try to be non-doomed with regards to repetition of history.
Or they know how complex and error-prone it can be, and decided to spend their time on other things.
It’s good to know how that stuff works, how to configure a LB, install nginx, rack a server... the way being able to do long divisions by hand is good to know. But when you’re crunching numbers all day, it’s easier to use a calculator.
If I want to load test something for a day and spin up 20 EC2 instances and spin them down, I can do that with a script. Then I can see where my bottlenecks are and provision instances, load balancers, increased disk IOPS, etc. as appropriate and tear everything down I don't need.
But as far as "vendor lock-in", it's like developers wrapping database access up into a repository pattern just in case we want to change databases. In the real world, hardly anyone takes on massive infrastructure changes to save a few dollars.
On the other hand, there are frameworks like Serverless and Terraform to build infrastructure in a cloud vendor neutral method.
In 2008 we had racks of servers we were leasing and that were sitting idle most of the time just so we could stress test our Windows Mobile apps.
I've been developing professionally for 20 years and 10 years before that as a hobbyist. I know what a pain it is to get hardware for what you need when your company has to manage all of its own infrastructure.
Just setting up EC2 instances and installing software on them doesn't reduce the pain by much. Sure you're cutting down on your capex but you still end up babysitting servers or doing the "undifferentiated heavy lifting", I would much rather stand up a bunch of RDS instances.
As far as serverless, why manage servers at all when you can just either create a Lambda function for the lightweight stuff or deploy Docker images with Fargate? That's just one less thing to manage and you can concentrate on development
All the parts are easily replaced or scaled however you see fit. Your function can be in any language that can respond to http on any platform you want. You can put whatever proxy you want in front to define your routes. You can get as simple or complicated as you want.
Serverless is not serverless, you are just abstracted away from it.
[EDIT] I would add that personally I would spin you a cluster of Flynn on Digital Ocean :)
And lambdas aren’t just about responding to http requests, they are also used to respond to messages, CloudWatch events, files being written to S3, etc. I would hate to have to stand up servers for that. Even if you don’t want to get “locked in” to Lambda, why not serverless Docker?
I'm all for moving forward and using new stuff if it makes my life better, but from the looks of it, Serverless is still many years away before discussions like that can even take place.
I prefer to use the label "FaaS", because it fits clearly into an existing taxonomy and doesn't spark silly arguments about what is, and what is not, serverless or serverful.
Disclosure: I work for Pivotal on a FaaS project.
This could imply that serverless "just need to improved" to a point is good for most.
I argue will NEVER* be good, not matter what kind of glue we put on top of it.
Because, I think, not matter what you do or how hard you try, is the most complex way to do a software project.
*Never: probably not in the next millenium....
I strongly disagree, yes it is early and some of the tools notably debugging and logs aren't anywhere near the level they need to be.
I'm developing an app with serverless and this article really resonated with me about my struggles. I think once Aurora Serverless launches allowing developers who need a relational database to easily move on the platform you will see rapid growth with serverless.
Why? Because it makes so much sense. Why worry about managing servers or scaling? Why continually write the same boilerplate glue code over and over again?
I know that I'd rather write a configuration file calling best of the best components over writing code. Don't get me wrong I like writing code but I'd rather concentrate on the business logic.
Lambda? If so, what benefit do you see over running AWS Container Service?
I ask because I've tried both. Serverless frameworks (AWS Lambda/AzFunc) were horrific. I picked up Docker as an answer and never looked back.
Others in my company are abandoning serverless after seeing our success. Turns out being easily able to run things locally, very similar to production, is very important. We have no problem concentrating on the business logic AND keeping flexibility.
I've loved my time with serverless framework and aws lambda.
Parents point is saying exactly that - filling out the toolchain to a meaningful completeness is going to take a couple of years.
This isn’t a ding on Lamdas - just a reality - theres a huge backlog of capabilities to catch up on.
Stop trying to write a full server and then map it to lambda. Start with lambda and map it to your service. There, done. That's all you need to know.
I just threw out my back cringing.
Dynamo is pretty good, but its value starts to dwindle when you want to be able to do local development, possibly even without a network connection. And most of the traditional ways of interacting with the data layer aren't really available. So for instance, you're not doing to be able to use an ORM for a simple application with Dynamo, which means writing a lot of your stuff from scratch.
So given that pragmatically, you probably still want a database, you're going to run into a position where you can't possibly be 100% "serverless". A persistent database connection is a good thing, and one where you can control the number of connections is an absolute requirement at scale. Even if you can tweak your lambdas just right to accommodate your DB's maximum number of connections, you're needless assuming the cost of opening one of those connections on each invocation of your lambda.
My recommendation is to use serverless where it's really well suited, which is for distributed, event-driven processing. Your data backend becomes an RPC that can help work with the top of the funnel to map and distributed well-populated messages through your system. For this, I use protocol buffers, and base-64 encode their serialized bytes into an SNS topic. Depending on message size, your mileage may vary here.
You can still use some of the more clever AWS offerings to reduce your dependence on some fixed, running server. For instance, Fargate may make it possible for you to run a persistent RPC server for managing read and write requests to your RPC, which is maintaining a well-optimized connection pool with your database.
I agree with using JWT for authentication. I agreed with it when stateless authentication pre-dated the service offerings that made serverless a possible paradigm. Serverless generally requires stateless, but you can still reap the benefits from doing the same thing with servers.
Hosting a static SPA in S3 I think is one of the less challenging arguments in this blog post and has been a good practice for getting on five years now. Vue isn't necessarily part of it, and marrying the framework with the choice of hosting I think muddies the waters on what's good advice and what's just an opinion.
In all introducing serverless technologies to your platform is a great way to significantly minimize your infrastructure costs. t comes at a similar price as building any other SOA -- an increase in the cost of maintenance as debuggability becomes more difficult, and the network becomes more complex. So it's important to think critically about what parts you should take and what parts you should leave or just defer when it comes to your architecture and your business requirements.
What the hell does this mean? Completely unsubstantiated nonsense.
Hardly anyone needs to scale Postgres past 1B records and 50K QPS, which can be achieved on a relatively affordable pair of synchronously-replicated boxes.
This guy clearly doesn't know anything beyond year 1 basics and the post reads like a fatal overdose of Kool-Aid.
It turned out really well and am very happy with it thus far. I expect maintenance to be minimal thanks to serverless.
Though I don’t think it’d work as well for an app with 100+ database tables. Definitely a middle ground in there.
I work with a platform that also does CRM/POS/ERP (plus a bunch more), and the products alone have over 10 tables for describing them: the base products, their variants, a list of generic variant attributes, a table detailing which variants have which attributes, a table for specifying the values that those variants have of those attributes, the product categories, two tables for configuring the taxes applied to each product (on purchases and sales), the product images, the list of suppliers.
We're already on 10 and we haven't even used those products for anything (stocking, selling, invoicing, purchasing, etc, etc).
I build huge serverless backend applications, and IMO it's been fantastic. There is a learning curve, because your application's execution environment is pretty different from traditional applications, but it's allowed us to build a remarkably complex and scalable application, very quickly. And it's been pretty maintainable too.
Basically we have an API that performs asynchronous data analysis and processing. Our fronting service receives a request, writes some metadata, and places the request in a "queue", which is picked up by a backend poller that starts a workflow execution, which orchestrates the fulfillment of the request. This is all serverless (AWS tech...API Gateway, Lambda, Step Functions, S3, DynamoDB, CloudFormation, CloudWatch, etc.). Serverless makes deployments much easier, since we can version our Lambdas and State Machines using CloudFormation, and have many different versions running at the same time (not fun if you're managing your own hardware!). We have a CD pipeline that builds code changes, deploys them to a test account, and runs integration tests. We use CloudWatch Alarms to monitor production and alert us of any issues.
We have some development scripts for pushing code changes with CloudFormation for testing during development. We use that to develop, then once we check the code in, it works its way through our pipeline and into production.
"That’s quaint" ... ugh. You're quaint.
> And now we no longer worry about Python version 2 or 3 (is it ever upgrading?)
*Node version manager
Just like Python.
Breaking news: your app still does this. You just moved the responsibility of request/response elsewhere.
https://read.acloud.guru/does-coding-language-memory-or-package-size-affect-cold-starts-of-aws-lambda-a15e26d12c76
Java with Spring is really difficult with AWS Lambda because of the slow cold starts. Five seconds for a cold start is unacceptable for many applications.
Except I opt for Golang and the Serverless Application Model (SAM).
Go let’s you ditch even more stuff by cross compiling binaries.
And SAM is a framework built by AWS and vastly simplifies the config files.
The site gives a cost estimate and an application score comments (latency, burstiness, function execution time)
I'm curious if anyone is actually using Cognito in production. It feels like an alpha product to me.
1) Is there a problem with python, or a problem with flask? Isn't this what chalice is for? https://github.com/aws/chalice
2) How are you dealing with cold starts?
I learned stuff from this post, but I would have learned more with some background about the workload etc so I could reason about what generalizes and what doesn't.
Edit: forgot to say thanks for suggestion! Also, here is a related article: https://hackernoon.com/im-afraid-you-re-thinking-about-aws-l...
How many instances do you want running? You could set up a separate keep alive path that sends another request to the lambda, with a variable on how deep into the keep alive request 'recursion' and break out if you deep enough. Does that make sense? Super weird and just off the top of my head.
edit: this isn't a good solution either because if you have a lambda kicking off 4 other lambdas because you want 5 running, and someone makes a request well then you still haven't warmed up that 6th and your 5 lambdas are running the keep warm code...
Just found the same author as OP with a clever solution here: https://read.acloud.guru/cold-starting-lambdas-2c663055589e
Having the app pre warm instances on a per-user basis is super cool -- for user-driven workloads like web servers. To make matters worse, we are serving an API that takes hits from third party streams -- so our concurrency is based on their client behavior, not something we can easily link to a session scope, like users. Tricky!
Sometimes though, you can't force a square peg in a round hole. I dislike server maintenance but docker is a decent alternative to lambdas if you can absorb the extra cost.
I'm curious whether the 70-90% savings include dev time as well?
Also looking for any gotchas, someone mentioned stability/compatibility issues in another post
Lambda serverless is really good if you have low RPS and want to pay-for-use due to the low RPS ( prototypes, small production apps, cron-jobs, regular scheduled events, compute intensive - image processing jobs)
I feel bad for this team and the future they’re going to face with the poor decision making at the top.
I wouldnt worry about the devs in their team so much, it’s great resume building, but I feel for the rest of the company.
Maybe someone who makes poorer technical choices than you doesn't deserve respect—that seems dubious, but we can argue about it. The community you're posting to, however, certainly does, and by posting like this you're not only disrespecting it but destroying it.
HN is a large, diffuse online community, so the bonds here are inevitably weak. Agitating snark acts as a solvent on those bonds, making the community less cohesive. This is exactly the opposite direction to the one we need. All the default forces already point that way; please don't make them worse.
I never said I didn’t respect the person. I said I didn’t respect the information in the post. I really can’t imagine how my remarks suggest that I’m a jerk.
Until I got flagged this was the top comment on the thread. The community seems to agree with my remarks.
I stand by them.
You can't judge this by upvotes. Indignation and snark often get heavily upvoted. That's a bug in the voting system, not an indication of comment quality. HN can't live by upvotes alone.
I mean, it's wonderful that you went to a conference and was able to take some interesting lessons that you could apply to your product, but coming back and dropping your entire platform to rebuild in a new language just to use the fancy new things you learnt is utterly insane.
But hey, Kubecon has just finished, so I look forward to the upcoming article "How I rebuilt everything around Containers", should be thrilling to hear about how you tore up your product and rebuilt it a third time based upon some cool conference talks.
Comments like these damage HN far more than a weak article.