Writing Cloud Functions
cloud.google.com
cloud.google.com
Lambda has a lot of strengths, and it gives you solid primitives like immutable versions, version aliases/pointers and stateless functions. The problem I had was that it didn't give me any more than that. Here are just a few of the things we had figure out and then build ourselves:
1. The development workflow. How can I get a REPL workflow going that doesn't make me go crazy.
2. Deploys and Rollbacks. How can we safely deploy and rollback, especially in cases where you have the same lambda function in multiple regions and each region is at a different published version (because the version number is monotonically incrementing)
3. Permissions and Policies. The broad strokes are clear. But you want your S3 bucket or SNS topic to trigger lambda? Get ready to spend an hour trying to figure out what you've done wrong and what's missing from the vague directions. Hope you get that REPL flow solid first. In the end, we built several scripts (create_build, publish_version, promote_to_prod, etc) and we use these directly during development and from a Fabric-based deploy script. When I have time I plan to release this tooling open-source.
If I had to do it all over again, I wouldn't. And I wouldn't use Cloud Functions. I would just use a t2.small instances with a simple worker.
I could not find any place where I can tell it - this is the code, this lambda uses it with N=5, that lambda uses it with N=42.
The best approaches they were able to suggest were:
* Lookup config from dynamo db when the lambda runs
* Lookup config from S3 when the lambda runs
* Bake config into the artifacts
Not ideal.
Some questions if anyone at Google knows:
- What are the CPU, memory and disk limitations? Lambda has caps on all three (5 min, 1 GB, and 512 MB last time I checked). - Pricing? - Native module support? And if so, what platform to target when creating our build scripts? Possible to download npm modules with pre-build binaries?
- html and js files hosted on s3
- api gateway talking to dynamodb
- no servers and worries about scaling.
The only downside thing is that it's truly tough to figure out how to point your domain to s3. I followed everything and DNS is okay too yet the domain never resolves. Tested with route53 and after 1 month of trying gave up and using digitalocean's DNS manager.
I implemented a token based HTTP Api, a fully functional Angular.js app that talks to it. Allow user to register, change passwords, login, create a blog.
It definitely requires new type of thinking: throwing shit out. Seriously, I've built apps with Flask, Node.js, variety of PHP frameworks on Apache, Nginx, Java Web applications and I think that a serverless architecture blows everything out of the water.
It's nice to know there's not a server anywhere. All of the HTTP requests are sent to AWS, and AWS runs a script that modifies or does things to the dynamodb. It feels me with so much excitement. I've yet to stress test it but I expect it to be highly scalable as AWS ramps things up.
However, I don't think I will be using AppEngine because I'm so used to AWS now it feels like a lot of cost to switch over to Google....I'm afraid AppEngine is another Google+ moment.
Guys, I really think serverless architecture is the way to go, and is the future!
I mean, really how much is Lambda more serverless than Heroku has been for years?
With Lambda, you're not getting a server. No OS setup, no ip-tables, no weird compile errors, and no 5am pagers. You're paying for code to run.
With Salesforce, you're not getting software. No installation, no packaging, no purchasing updates. You're paying for your team to centralize information.
While I despise Salesforce for many reasons, it was a game-changer in the way enterprises thought about cloud software. GP's point was a similar mental shift in how we write that software.
It seems like an ill-fitting adjective even in a vacuum, but what's worse is that "serverless" is commonly used to describe something that runs entirely client-side (e.g. p2p).
Just because AWS Lambda is a "mental shift" doesn't mean that "serverless" is a good way to describe it.
As for a better term, perhaps operationally-opaque or "pure service" are starting points? Service is correlated with stronger federation while p2p is decentralized as a rule. I'd rather avoid SOA which does not say anything about whether you have to manage the operations or dependencies - nobody is calling microservices as "server-less" after all. When people in companies talk about a "services" company they mean that the labor and overhead of management are all outsourced and as a customer you don't really care about how they operate as long as they don't violate certain OLAs and SLAs. AWS can run Lambda on marmot-based computing on Mars for all I care as long as it meets certain specs. Services compete on cost, SLAs (features are part of SLAs), and OLAs, period.
But there's another interesting thing that is relevant here. Because it is stateless, it it really is "serverless" in the sense of there being a single server associated with your code.
They really could run it in disposable VMs ZeroVM[1] style.
Not having to worry about anything else except the code opens other doors.
It should be fairly straightforward. In fact I just did it this morning. What issues did you run into? One tricky part that a lot of people miss is that you have to choose Alias: res in the record set properties.
To be clear AWS is running thousands of servers that make up Lambda, DynamoDB, S3 and friends. It's still fair to call Lambda "serverless" because you don't have to deal with any servers... but saying there are no servers anywhere is false.
*edit note: looks like you can definitely add more files via the package.json and import them as node modules, but I wouldn't rely on reading and writing from the file system.
As for persistency, if it's just Javascript code you can install node modules to accomplish what you want including talking to those cloud APIs. Unfortunately Bigtable might be the exception since I'm not sure if there's a good nodejs client yet (Bigtable only supports gRPC, not REST), but you could use Cloud Datastore instead.
Firebase will also be a very popular choice for persistence. Cloud Functions naturally complements Firebase to complete Backend-as-a-Service. The hope of Firebase is you don't need a server at all, but sometimes you have logic that really doesn't belong on a client (background jobs, "master" logic). Cloud Functions can fill in those gaps.
Firebase: https://www.firebase.com/
hand-rolled clients nodejs clients for google cloud apis: https://github.com/GoogleCloudPlatform/gcloud-node
auto-generated nodejs clients for google apis: https://github.com/google/google-api-nodejs-client
disclosure: work for GCP
I wasn't aware of those libraries for accessing Google cloud sql - so thanks for those.
After playing around with it, I'm not sure if the filesystem works, which makes sense given you just have a bucket or repo underneath and the code can be called at unpredictable times. You can, however, add additional javascript files via the package.json and import them, so if you can express your data as JSON or base64, you're good to go. Maybe there is a way to add other types of resources via the package.json but I haven't figured it out yet.
Generally you would interact with Cloud SQL using a regular node MySQL library, since it's just managed MySQL. The API would only be used to automate creating and setting up the CloudSQL instance.
Give me something that competes with Heroku.
But with more noSql Databases like RETHINKDB and CHEAPER noSQL DB
and CHEAPER SSL per month for non-wildcard
It's not 100% apples to apples, but Google Container Engine (hosted Kubernetes) is young but already outstanding if you are comfortable with Docker. If not, nevermind me.
I see some people using it for small use cases and it worked fine. In particular "I need to run this tiny integration function somewhere". However, I would be really careful to do something that has huge traffic.
But other than them I don't know of anyone else.
My company is built totally on Lambda, but we haven't launched yet.
Not that I care to have that level of introspection into my data but it opens up some interesting opportunities.
Disclaimer: I work for Google.
I.E. when would you use this over a plain old simple appengine app
These will allow you to automatically scale up or down easily. You don't have to maintain your server.
It has its advantages and disadvantages.
Considering that the lambda function was pretty easy to write, free to run, and very portable: why would an Appengine app be better?
I think the general argument is this. Javascript is a necessary evil, but serverside, it isn't. Please, let us use something saner, or at least give us the option to do so.
Javascript seems to be spreading into areas javascript has no business of being in, ("node.js on microcontrollers? what?"), and that is a bit disconcerting.
Granted, I'm fairly new to Javascript, but even the good stuff seems like piecing together sane abstractions from wonky components. It's the horror you'd feel if a nuclear reactor operator told you they get the components to go subcritical by grounding that bare wire next to their coffee cup. Does it work? Yes. Abusing function scope and closures to make variables private? Write a function that checks whether a variable is an array because typeof malfunctions? Does it work? Yes.
But then, is it sensible engineering? And even if you've gotten good enough at javascript that the gotchas simply seem kinky to you, do you really need it in every other area of your life?
I'm also be a little bit concerned that javascript will swallow everything else, and then the only way to program is to engage in a kinky whipfest, and whilst as a German I enjoy a good kinky whipfest, it's really something for after work.
I think the fear and distrust towards it stems from the perception that it is on its way to become the lingua franca of development, and the expense of other much more suitable programming languages. That seems like a terrible idea.