Going Serverless: Migrating an Express App to AWS API Gateway and AWS Lambda | Hacker News Reader

Going Serverless: Migrating an Express App to AWS API Gateway and AWS Lambda

aws.amazon.com

145 pointsdavidjnelson133 comments

Serverless via Lambda has been, frankly, a disappointment so far. The benefit is that I'm supposed to not have to manage servers anymore, yet I find myself, well, managing servers.

I have to build and assign IAM roles, subnets, set up application configuration files in fixed external services (since you can't do things like set environment variables), configure the endpoints in nginx^h^h^h^h^h API Gateway, suffer through cold starts, fight against arcane packaging issues with site directories and .pth files (though this is probably just be a Python thing), fight concurrency throttling, external timeouts, manage out-of-band database connections, logging which requires even more IAM permissions...

This doesn't even consider how many times I've had to tear down and re-create an API Gateway setup because it got "stuck" and would stop working.

Finally, WTF is up with not being able to test API Gateway -> S3 integrations if the S3 bucket is in the same region as the gateway instance? It's a two plus year old bug by now, and a real pain (especially when coupled with other AWS services which require the bucket to be in the same region as the rest of a service - such as CodePipeline).

Perhaps it's just because I'm familiar with setting up and provisioning servers, but for basic web services and periodic tasks, Lambda is much harder to work with most of the time. Its biggest benefit so far has been the low cost.

> Perhaps it's just because I'm familiar with setting up and provisioning servers

I'd be inclined to say that may be the case. Our codebase leans heavily on Lambda, primarily as a wide stage in our data pipeline. And I love it.

Using a configuration file in S3 - or any other means of dropping a text file on AWS - is barely more complicated than open("file.txt"). The true frustration is when you need to work with files beyond your memory limits, but even that can be mitigated by divide and conquer approaches.

That said, the API Gateway is annoying in that it makes complex things impossible and simple things more configuration than they need. Yet, I'd still take it over actually standing up a server.

> is barely more complicated than open("file.txt")

Well, that depends. First, you have to set up the IAM roles to access the bucket which owns file.txt. Then you have to ensure that "file.txt" is properly shared between accounts if you separate production from staging from development. Then there's three lines of code to set up a client, pull the file, and read the contents. If it's an encrypted file, you also get to do a bit more setup work to ensure that you're using the v4 signature in the library, since it defaults to v3 signatures. There's also the additional error handling and points of failure to account for.

The requests are far from instantaneous as well, so you need to set up an out-of-band cache for the contents of the file if you plan to use it with every call of the lambda function.

Is it obtusely hard? Of course not. Is it as simple as `open("file.txt")` (or more appropriately `os.environ["FOO"]`)? Not by a long shot.

I have a feeling that our differences in opinion have more to do with the environment in which we run our lambda functions than what the lambda functions are doing. Once you move beyond a single region on a single account, things get a lot harder.

I may be overly simplifying working with a file, but I'd still argue you're over-complicating it :P You're hitting on a good point with Lambda though, namely the stateless aspect. To get the most out of Lambda, you have to adopt a more Functional-based programming model.

> Once you move beyond a single region on a single account, things get a lot harder.

I could see that.

I'm working in IoT, primarily connecting lots of dumb devices to small translation layers with minute data massaging along the way. In this type of low-intensity environment, the flexibility Lambda (AWS in general) gives you is tremendous.

The second performance(/latency) becomes a primary concern, it's easy to see that trade-off become untenable.

@hobofan, take a look at our recent API Gateway features and see if the greedy paths and pass-through settings provide what you're looking for (they're also supported by CloudFormation). We tried to simplify the "configure every route" problem, but always looking for additional suggestions to make API config easier.

@jomamaxx (& others): Reducing latency (and latency variability) is a critical goal for our team. We've improved p99 variability in API Gateway-related latency over the last few months, and will be addressing some of the Lambda-related latency that occurs when we refresh containers in the coming months. Additional latency optimizations coming at all levels of the stack, including networking.

A clarification on the discussions about "managed hosting": Lambda is not classic web hosting; in fact, we block Lambda functions from calling socket.listen. In the Lambda model (whatever you prefer to call the broader category), the cloud service sees every request in order to perform scaling and load balancing on the function's behalf.

Happy to chat offline with anyone who has additional questions or feedback: DM me on twitter @timallenwagner

Tim, when are we getting Python 3 support on Lambda?

Being stuck on 2.7 has been the greatest source of issues and the #1 problem we have with Lambda. If Google/Microsoft came out today with Python 3 support in their Lambda competitor, we'd move in an instant.

Piggybacking above comment with "Wheres da Ruby?" :D Using shims or jruby (with slow startup wasting milliseconds) hurts.

I'm in the same boat. We're developing a new product with no real reason to stay on Python 2, besides Lambda. Python 3 support would make my life so much easier.

Yep, I hear about the lack of Python3 all the time, and I know our Python users are waiting for it. On the roadmap, and partially complete, but competing with some other language work at the moment.

The point is "serverless" is a sham. Your clients still need to rely on another service API Gateway to communicate with the outside world. So stop with this stupid marketing.

Your are selling a basic pubsub architecture as something "innovative" when the only real innovation is the pricing. That's the exact definition of marketing, and at the same time you are making the community of web developers sound like idiots that don't understand what a server is. You can make plenty of money without being missleading, why do you feel the need to use a misleading language ?

There is a server, period.

Of course there's a server, or do you think you're the only genius in the room who understands client server communication? "Serverless" is a legitimate phrase to describe a growing phenomenon. Most reasonable people read "serverless" as "server I don't have to admin", not "there is no server". No one wants to read your pedantic bloviating, get over it!

> Most reasonable people read "serverless" as "server I don't have to admin"

Most reasonable people find the "serverless" moniker obnoxious to begin with. If something doesn't require administration then call it "adminless", not "serverless"

As others have said, serverless applies to the pricing model, in which it makes perfect sense, imo.

Yes we agree that "serverless" is pure marketing. as I already said.

> Your are selling a basic pubsub architecture as something "innovative" when the only real innovation is the pricing.

I would want to know, how do I manage user credentials in AWS lambda. Lambdas are stateless, there is no session, so how do I keep track of which user is executing the current lambda?

How do you do it with a load balanced stateless web server? People have been doing this 10 years, most language should have some frameworks or plugins around it....

Common is to give the client a cookie with a session id (KJASDJKASDASDS) in a cookie. Then in a RDBMS you store this (session KJASDJKASDASDS = User 1234)... and the first part of each web (or lambda call), you go look up the user by session and know who it is.

(You can also store the user ID directly in an encrypted cookie, but that has a few other problems)

Thanks for the reply. In which DB should I store this data, considering the option of either DynamoDB or ElastiCache?

Why not MySQL or PostgreSQL? Any reason you can't be relational?

Because I don't want to manage my own DB, so I would rather stick with the options that Amazon offers as a service.

You realized Amazon offers hosted MySQL and PostgreSQL, right? One click, and it's up.

https://aws.amazon.com/rds/

Ok, I want a 2 pronged approach:

1) A NoSQL DB for the main data providing replication and scalability(e.g. Cassandra or DynamoDB).

2) Another DB for quick access and transient Data where replication is not so important, e.g. storing the session cookie. Relational vs Non-relational would not be an issue since I'm only storing very little data here and want fast access and minimize costs(in DynamoDB you still have to pay per operation). Therefore I'm looking for some simple solution like ElastiCache.

How much data are you going to have, and of what format? Do you actually need NoSQL here?

I'm doing a social networking site, so I don't know how much data I will end up with. My reasoning was as follows:

1) The data model is rather simple, basically one table with the user data(name, location, age, etc...) and some images. So it shouldn't be too difficult to do this with NoSql as opposed to SQL. Yes, simpler with SQL but not significantly so.

2) If I ever need to scale, I'd rather make the right choice from the beginning(NoSQL) so as not to need a migration later.

3) In any case NoSQL would be a learning experience for me, so another plus.

If there is a mistake in my reasoning I would be grateful for any one pointing it out.

Just curious, what would be the major drawbacks of not using relational here? My understanding is that AWS is pushing Dynamo quite a bit and it seems cheaper, so if they're only needing to query off of session id and maybe user id, shouldn't that be sufficient?

Relational is super handy. Very easy to report on and do various things down the road.

If all he needed was session id, sure, NoSQL is more or less the same. But what about when he adds other fields that are related? Say customer address or reports or .... His life may very well be easier with relational.

You should generally start relational. Then branch out if you are hitting the brick walls of relational. People using NoSQL for a 100MB database are making their life SERIOUSLY more difficult than it needs to be. (I have done it before, not fun).

> You should generally start relational.

I don't know if it is that easy to scale once you realize that the demand is growing, see my other reply here:

https://news.ycombinator.com/item?id=12692234

I'm always surprised at the lack of discussion around database connection reuse with AWS Lambda.

It's a pretty big deal that every single API call requires a new database connection. The only solutions I've seen so far are to run a separate app to interface with the database, or moving the connection outside of the handler (which still has issues).

Am I missing something or is everyone just really happy to use DynamoDB?

If you design your correctly (at least in Python), you can cache and reuse your open database connection.

How exactly have you been doing that? It would be a cool blog post if you had the time. I've been thinking about really high volume applications that connect to haproxy instances that do DB load balancing but even that has it's own set of initial latency issues. I haven't fully figured out that architecture. In python in particular I'd really love to know how to setup the connection reuse.

Check out the handler source code in Zappa for an example of a pattern like that. Similarly, if you use Zappa to deploy your application, if you create your database connection when the application loads, it'll just work.

Stop by the Zappa slack if you want to explore this in more detail! https://slack.zappa.io It could always use more investigation, but there are quite a few Zappa users at extremely high loads now without any issues.

This project looks super interesting. Thanks for sharing it! Now I want to come up with a use case to test it out over the next few days, maybe something for the Echo? I joined your Slack, we'll see where this takes us :)

That's a great use case, many people are using flask-ask and Zappa together to great success: https://github.com/johnwheeler/flask-ask

You've got a problem with the naked domain redirects http://zappa.io https://zappa.io/

One way to do it is to stand an http rest API middle layer in between with something like http://postgrest.com.

If you are creating connections outside of handler, it gets cached by Lambda between invocations. However you'll need to tune your database settings or you'll run out of connections on traffic spikes. Depends on your pattern of usage. Also cold start of lambda in VPC could take around 15 seconds https://www.reddit.com/r/aws/comments/49l91l/lambda_function... https://forums.aws.amazon.com/thread.jspa?messageID=735318&t... My experience with DynamoDb was much smoother than with Postgres RDS in VPC

I am curious what sort of stuff is being built that need microservices (presumably so they scale) yet still has a database back end (usually the IO is the bottleneck in most applications).

DynamoDB is not immune from that problem. There is no way out: if the container is terminated the connection goes down. The key is not making Lambda stop and rm the container (in docker terms). If there are enough requests the container is reused and the connection stays up, but you must initialize it outside the function. An example with DynamoDB

    const AWS = require("aws-sdk");
    const docClient = new AWS.DynamoDB.DocumentClient();
    module.exports.theFunction = (event, context, callback) => {
      var params = {
        TableName: "table",
        Item: {
            "attribute": "value"
        }
      };
      docClient.put(params, function (err, data) {
        ...
      });
      callback(null, {...});
    };

I didn't try but it should work with any other db.

However, is Lambda cost effective for services with the amount of requests required to have their containers almost never terminated?

If there are enough requests to keep your container up all the time, why are you using lambda??

Exactly what I was asking but I didn't do the math.

Maybe it's OK for burst shaped traffic. The first request pays a toll, the others are quick.

I think Lambda was never designed for building entire apps/APIs - It was designed for handling mid-sized processing tasks.

Is application side connection pooling always necessary to have good response time?

MySQL has a really fast server side connection pool.

Usually 15 or 20 ms hit to make a new connection even in MySQL.

Also adds a fairly big server load. I suspect if a given MySQL instance supports 1000 long lived connections.. it would only support 100-200 connections that are closing every request. (Have not benchmarked this side, would be curious to see )

Hi Everhusk, I don't specifically call it out in the article, but the code is actually reusing database connections. As mentioned in other comments, the trick is to establish the connection outside of the handler so that the connection only happens once per container.

If anyone's interested in doing the math about the costs involved, I have two quick notes: first, the API Gateway, at $3.50 per million requests, is typically more expensive than the Lambda functions themselves (and comes with no permanent free tier, unlike Lambda); second, the 128MB minimum memory for a Lambda function can be on the tight side for an Express app, so you might want to base your calculations on more memory.

I've recently deployed a small Flask based micro-service for an internal tool leveraging Zappa[0] to deploy it to Lambda/API Gateway. Overall it's been a really smooth experience. I'm looking forward to using it again.

[0] https://github.com/Miserlou/Zappa

Is there a big overhead in terms of latency? If I understand correctly you essentially have to start a server for each request.

~20ms, and there is no parallel burden, so it's usually faster in medium to high traffic scenarios.

Not the case - it runs with all dependencies cached if its called multiple times. Inconsistent calls will incur the delay though.

Zappa author here - thank you for your support!

I love zappa, there's also chalice: https://github.com/awslabs/chalice

Have you used both? How would you compare/contrast them?

So, Apigee had this exact service. They did something pretty cool, it seems they implemented nearly all (!) of the Nodejs APIs in a JVM variant, then designed them to run on demand [0].

I actually ported an Express app to Apigee's service, to run webhooks and ETL. It worked flawlessly. The one downside was that they kept their own internal registry of compatible packages and there was no way they could keep up with the Nodejs ecosystem even with few holes in their API.

I still have a ton of hope that Google does something smart with this, I actually wanted to pay Apigee for their service, but their pricing started at an Enterprise tier and none of my projects made it that far.... yet.

Something to think about.

[0]: http://docs.apigee.com/api-services/content/overview-nodejs-...

Wherein "serverless" basically means more computers in more places just doing different tasks.

Some people prefer to call it Function as a Service, which in most cases is closer to what it actually is.

"serverless" does not refer to the absolute lack of servers - it refers to the fact that your CODE (applications and software in general) should not be AWARE of being executed on a physical or virtual machine.

The platform should be as abstract as it gets. I always imagine it as coding on a piece of paper: in a way, my code should work and make "sense", even though we all know that a piece of paper won't execute any instructions. And this is part of the magic of "serverless" software: it imagines a perfect, abstract machine where it can run, forgetting about the complex, unreliable, physical machine underneath.

It's an almost philosophical approach to writing software and getting it needs some time and thought. And needless to say, it does not solve ALL your problems, but it MIGHT solve SOME of your problems.

> it refers to the fact that your CODE (applications and software in general) should not be AWARE of being executed on a physical or virtual machine.

With the exception of para-virtualised device drivers, what code is ever aware of being executed on physical or virtual machines?

C'mon now, you know what was meant...

No, I honestly don't, using that definition.

AWS Lambda and similar services have literally nothing to do with hardware vs software.

They're about running a function in response to an event on someone else's stack, almost like the `map` part of a `map-reduce`, if the input was considered to be distinct events/objects.

Or just RPC writ large.

I don't think it has to do with the way the application is written at all. It's how it's deployed and managed.

Yep. It's good old fashioned webhosting. Drop your files in the amazon equivalent of ~/public-html/ . If it serves your needs, it's not a bad thing, but it's hardly revolutionary.

Is there any other service where I can run arbitrary code and be billed in increments of 100ms or less?

Webhosting usually have a flat monthly fee so maybe Lambda it's more like old fashioned mainframe timesharing, billed per resource usage. Not that I was there in its heydays but the two concepts seem very similar.

A consequence is that you really want to terminate quickly and use little RAM. If your JS function terminates in 110 ms and get billed for two 100 ms units you might be tempted to switch to a faster language (or ASM what you can) to save half the bill. Webhosting doesn't have that dynamic.

How do I setup "old fashioned webhosting" to respond at-least-once to events posted to a message queue?

How do I get "old fashioned webhosting" to automatically scale-out and scale-in?

How do I do all that and get per-100ms CPU-time billing?

A little of a shameless plug, but I am the author of effe GitHub.com/siscia/effe A microservice framework.

If you are worried about latency or you want lambda on your own servers it may be worth to take a look.

Feedback welcome

One significant problem with the API Gateway / Lambda integration is that the protocol they use to communicate is JSON. JSON is a text-only encoding, i.e. anything that isn't valid UTF8 isn't valid JSON, and so if you need to pass something binary to/from the Lambda (e.g. an image), you need to base64 encode it, which I'm actually not even sure is at all possible with the API/GW templates. There are other issues, but this alone is a huge headache if you ever have to work around it.

I hate the term serverless. It involves a server. AWS Lambda is a service bound to a port therefore a server. It runs user defined functions, true. But it is a server.

Isn't it more of a cluster rather than a server

A cluster of? servers. Again, server. server, server, server. Repeat with me: server.

It's just semantics, point is, you don't have to worry about spinning up servers, provisioning them, or maintaining them; just write code.

Write code to run on? a server.

Call it whatever you want, but I like to use the language of those who hold the purse-strings. If they call it "serverless", then I call it "serverless". If delivering a lecture about why it's not really serverless improves your product, then go ahead and deliver that lecture, but I doubt it will.

Involves a server listening on a port, running on infrastructure that you indirectly pay. Not in the air, on a server.

It is a very misleading term. Especially for non-tech savvy. Sounds like a scam.

It's like calling being passenger on a car "wheel-less transport", because I as passenger don't have wheels.

A driverless car has a driver, it's just not a human driver. A serverless architecture has servers, but you never see or need to think about them.

Don't get hung up on the name. Bananas are not a fruit, but it's fine to put them in a fruit salad.

It's more like driverless cars. Something clearly is driving the car, but it isn't you.

Moreover the fact it's even a car is now an implementation detail. It could just as well be a motorised shipping container that slots into a hyperloop when it reaches the station. The assumption that the AWS Lambda infrastructure is server hardware is just that - it may not be true now, and you could build a function-as-a-service platform out of a massive array of Raspberry Pis and it'd still have solid at-least-once invocation semantics.

And I'm repeating this, but you're flat wrong about the listening server-on-a-port. AWS Lambda functions are bound to queues and receive events. The closest you'll get to having a socket-on-a-port is that the provisioning API has a REST endpoint.

So if it isn't necessarily server hardware, and there's no server endpoint to connect to, that "hate" you started this thread with looks rather misplaced.

It isn't bound to a port. AWS λ functions are event-driven. They're bound to queues.

It would help if people referred to this technological concept as Function as a Service. That is a lot less ambiguous and buzzwordish than 'serverless'.

"serverless" is the pricing model as far as i can tell

Me, too. And "driverless cars," also.

There is a driver! It's just in software! The car's still got something driving it. It's just not a person!

People tell me, "Dude, that's pedantic. Clearly what people mean when they say 'driverless cars' isn't that there's literally nothing driving it. They mean that having people driving cars has a lot of downsides and having the benefits of cars available without having an in-person driver opens up a lot of opportunities and frees us from personally having to drive ourselves."

I hate it when they tell me that.

Autonomous car could be a better term.

You're right. There is a clear and accurate alternative in this case. What about for a serverless architecture?

I am currently using AWS API Gateway + Lambda for the serverless backend of a social traveling startup. It is absolutely awesome: we are few people in IT and it is amazingly simple to manage the whole stuff. Moreover imho they are mature services at this time, they are flexible enough with enough configuration properties to fulfill a nearly full control of the development experience.

How much does it end up costing you? Any ideas how much a 'traditional' setup would be in comparison?

We reduced our service server costs to 1/6th by moving from Heroku to Lambda. And that's not even considering the main benefit of a big computational request not blocking other requests (which is why we moved to it in the first place).

Can you achieve competitive latency for web pages using API Gateway and Lambda?

I think acloud.guru is served through lambda. I think if you go the single page app route with S3 + cloudfront CDN + API gateway + lambda (so many words/things) you can deliver a really high quality product. You can also cache responses out via elasticache. It's not perfect but the development velocity and potential is really exciting.

The site itself is powered by Netlify.

I was going off of this article: https://read.acloud.guru/serverless-the-future-of-software-a...

I was curious about the same thing and setup a jvm lambda to test. The setup was served through API gateway, connected to a postgres RDS, selected a couple records from a single database table and returned them as json, and deployed in us-west.

From the west coast, I'm getting between 150ms and 300ms response times for most requests, but it does jump around quite a bit, with occasional requests taking 500ms-800ms to complete.

A lot of the variability seems to be coming from execution time of the lambda itself, which would probably be because of the connection to RDS? I'm using HikariCP so the connection is pooled but maybe something like dynamo would be better.

So latency is all over the place, but generally under half a second.

Connection would only be pooled on a per-instance basis, which can be cached but sometimes are not for many parallel running processes (to my understanding).

How does this work, if at all, for local development? Is there a pseudo-Lambda server that runs locally?

If not, this introduces yet another environment variable or it requires changes to be pushed constantly to Lamba for test.

Check out our example on github https://github.com/awslabs/aws-serverless-express/tree/maste.... The lambda wrapper is very thin (4 LOC). So you have two options: run/test your express app locally as you always would, or use the provided `npm run local` command which simulates the API Gateway+Lambda part (you can modify the `api-gateway-event.json` to change the "request" from API Gateway). This is primarily an example and starting point, and there is much more you could do to improve this process.

I have just recently got through a migration from express to a lambda based stack. In the end, performance is pretty good (once the lambdas are warm), but there were a few tricky parts, but nothing impossible.

Serving backend requests is easy with Lambda, but the code for managing the backend can get quite large, quickly. If you are loading up data and it's going to take more than 5 minutes to perform a task, you need to build a queueing system from AWS primitives.

I hope they solve the latency issues for a basic function that returns text (a basic callback(null, 'hello world') with the api gateway that is running 'hot' with the container model and yet still has so much latency.

What is the benefit of AWS Lambda vs getting a cheap VPS or a dedicated server with much better specs and bandwidth, setting it up and adding auto-update? And how does one prevent going bankrupt if a Lambda app is DDOSed?

The cost is stupidly low. You would really have to work hard to go bankrupt.

That said, there is a concurrency limit, which would help prevent your server time going to exponential limits.

Article doesn't mention any numbers. OVH VPS is $3.50 for a 2GB RAM and 100mbs (about 14TB a month assuming 50% utilization). That is about 14mln pages with 1MB size vs 1mln requests for API Gateway at $3.50, not including Lambda costs or bandwidth costs.

I don't have exact numbers, but we have dozens of lambda functions written in Python which are parsing and acting on Kinesis log streams (at volumes that cause problems for the Splunk ingest), and regularly hit the 100 process concurrency limit... but our bill is well under $10 a year (yes, year, not month. Hence my use of the adjectives "ridiculously" and "stupidly").

Luck? Hidden discounts? Don't know. Just know that our bill is ridiculously low for everything we're doing.

Auto updates can break shit; you need to monitor and understand the output of the auto update script. That understanding doesn't come cheap.

i hope they upgrade to node 6 soon. i can't use fancy destructuring and some other es2015 functionality!

You can use Babel for server side code too which would get you what you want.

I love lambda's, but api gateway was far from mature the last time I used it (6 months ago).

Unless they have made serious updates to the deployment workflow, feature set, and documentation I would not recommend building your products on it.

They released proxy resources to API Gateway. Lets you specify a route like /foo/{path+} and all requests are forwarded as-is.

Great for simplifying universal web apps.

The "0 to 1" workflow right out of AWS could not be easier, they have an in-browser code editor with a hello world already written for you.

But that workflow doesn't make sense for real dev teams. So there's https://github.com/serverless/serverless . It's driven by a yaml file and is intuitive.

The only thing I'm bothered by is lack of environment variables (have to hard code them) and terrible API Gateway latency - see https://news.ycombinator.com/item?id=12681926 further down the page

For js projects you can work around that problem with

const config = require('./config.json');

Then you do config.secrets.apiKey etc

You have to gitignore config.json and include it in serverless.yml

Other languages might have to read and parse that JSON into one of their data structures.

Have you seen https://github.com/silvermine/serverless-plugin-write-env-va...?

Serverless was nowhere near ready the last time I looked.

Are they still requiring sn unrestricted iam profile? That is a non starter for most orgs.

Also, I found working with serverless to be pretty unintuitive. You are relying entirely on large config files and it was very difficult and time consuming for me to test and rollback several different configs.

I'm in the same boat. I have 4 microscervices set up with API Gateway + Lambda and now actively guide people away from API Gateway.

Do you have any feedback that I can pass along to the team? Post it here or email it to me (email in profile).

I'm in the same boat.

If I would have to name one thing, it's that the configuration of API Gateway via the UI is unbearable if you have a nontrivial amount of routes (> 3). I need to configure Authorization, Error code mappings etc. for every route, although they are exactly the same. If there would be per-gateway defaults I can set with per-route overrides, that would cut down the setup time by 95%, which is the main thing holding me back from using API Gateway for projects that would suit it.

Right now the only option I seem to have is using a tool like Terraform to automate it, which is a pain to adopt if you already have a running API Gateway.

You could also write a quick script using their REST API, although that's a bit cumbersome (lot of IDs that need to be passed around after creation of each resource).

Still better than manual clicking in my opinion.

http://docs.aws.amazon.com/apigateway/api-reference/resource...

You might want to play with https://github.com/awslabs/chalice

It gets you to hello lambda (python) in one shot, from the cli. Anything written down in one repo is going to be far more undertstandable than piles of hidden behaviors and obscure error messages. I think there's a new one on their blog for node.

I'd recommend checking back in for a hot minute every few months or so. The development velocity on this service is pretty rapid. https://aws.amazon.com/blogs/aws/category/amazon-api-gateway...

Listen folks - it's still a fantasy. Latency is still 5s on many requests = no deal.

This has been going on for years.

Would someone form AMZ please stand up?

I suggest that the latency is 5s for the 'first call', then, subsequent calls are fast - and if you don't use your lambda for a minute or so, it goes back to having latency.

I suggest maybe this has something to do with loading it into memory when it's used?

There must be a way to keep the lambdas 'hot' or else it will never really get used for anything but background tasks.

You break the entire model of an oversubscribed multi-tenant container system if you want every lambda hot all the time.

If that's what you need...you need a server.

"If that's what you need...you need a server."

We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again.

"You break the entire model of an oversubscribed multi-tenant container system" - we don't care that much about how it's implemented, or what underlying model is used, as long as it is secure, robust, and reliable.

> We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again.

I'm going to argue that's currently unrealistic, but if you want someone to build something, tell them its impossible.

Clearly its not easy to build a system that'll run code on demand as fast as bare metal; accept the trade offs with the understanding why you're making them.

What I'm asking for is definitely within grasp of Amazon's team :) they just need to get there.

Moreover, it's a rather fundamental opportunity for them: Lambda's are a form of 'container' and they are exactly what so many of us want. The deal-breaker for us was the latency issue.

Amazon is a pretty smart company, and it's a big opportunity, I'll bet they eventually get this sorted out.

What you're describing is managed hosting. In order to do it, you either have to oversubscribe like hell, and deal with latency, or pay for wide open swaths of server. Pick your poison.

Keeping a Lambda container in memory is probably not going to take more than 500MB of RAM. Paying a monthly minimum fee per lambda per machine per month to keep it in RAM should be an option. It would still be far cheaper than a real server.

Would you be able to explain why it'd be far cheaper than a real virtual machine? Because the ram is the most expensive part in both cases.

stands up https://www.youtube.com/watch?v=eJO5HU_7_1w

You're describing container re-use which is documented in the page here: http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduct...

You could have cloud watch run the function every minute? Again, that won't guarantee that your function will be "hot". But it's a quick hack to potentially reduce the frequency of your slow initial start.

That said customer feedback is super helpful in making great services so I'll get this feedback to the lambda team (or they may see it here).

If you ever need any help my email is in my profile and I'm happy to work with you.

Thanks for your response.

I checked out the doc and didn't see any reference to latency or how to mitigate it, perhaps I'm missing something?

Listen - I'm a massive fan of AWS. We use some core services and are very happy.

I know you listen to customers - but I suggest that Lambda was probably designed for specific kind of services (maybe IT, admin, batch processing etc.) - I'm going to fathom that maybe 'people running web-sites' were top of mind? Again, purely speculation on my part.

But if you want people to seriously switch over - and they will - in droves - if you get it right, is to basically enable fairly responsive services that scale with a minimum of fuss, latency concerns included.

So - if I'm running a web-portal - and I have a customer pull some data to build a chart from somewhere - and I have a Lambda specifically geared for that microservice - then it needs to be quick.

There really should be no need to use other services to have to hack around the latency issue.

Surely - you can parameterize the feature if you want, and require users to set thresholds for how long Lambda's 'stay hot' (and forgive my crude metaphors), or how much throughput etc. - and you could even bill them based upon those parameters. As long as it's fairly concise and understandable.

Perhaps DynamoDB is a good example. We've used it before - and what we love about it is that as long as we design our data correctly we never have to worry. Bursts of traffic, etc. are generally handled well, it scales largely without concern. One poor limitation of that service is the requirement to indicate throughput thresholds, which is too bad, but beyond that, it's ideal.

Amazon Lambdas are basically the 'last piece' of the golden dream of cloud computing - the ability to develop a truly serverless, microserver-based infrastructure, and never have to worry about platforms again. As much as we love EC2s, we would drop them instantly and go to Lambdas if they worked well. (And it's been at least 18 months since we last experimented, BTW). There are tons of SMEs that would do the same - I don't think anyone who isn't directly in the business of tech, and is providing some other kind of consumer or business service, wants the pain of managing EC2s or servers. 'Elastic Beanstalk' etc. is a bandaid - it's not ideal.

Anyhow - Lambdas are very close to being a super great thing, more than that - transformational - if they could just cross that 'real world' usability threshold.

Again - if something has changed WRT latency, and there is something in those docs I'm missing, please let me know.

Not having PCI compliance in Lambdas are a huge drawback that keeps us on EC2s despite having a large amount of our webservice now on Lambda. API Gateway latency would be a big problem if we were doing a high perf webapp. Fix those 2 problems and we get to do serverless all the way.

I measured the actual latency over the weekend [0] [1]

Bare ec2: 15ms

Lambda via API Gateway: 276ms

[0]: http://prerender.cloud/lambda-latency [1]: https://news.ycombinator.com/item?id=12678011

Thanks for the data points.

I understand that Lambdas are 'relatively quick' when they are 'hot' - which is my crude metaphor for 'up and running'.

The issue is, if the Lambda has not been in use for some time (seconds, minutes?), then the 'first call' can have quite a bit of latency - often up to 5 seconds - which killed it for us.

There is no way we could reasonably deploy a service wherein customer would have to wait an additional 5 seconds for a reaction in most use cases.

So close, yet so far.

This was about 18 months ago.

If you have some data that addresses this specific artifact - then that would be great. Possibly things have changed in 18 months.

And I'm not a hater, to us AWS is the best thing since sliced bread.

Actually I was making the point that they are not "relatively quick" even when hot: close to 300ms, almost entirely due to the API Gateway. (I even measured API Gateway on top of bare ec2 and the same penalty applies).

Lambda itself is fairly quick (~75ms), but API Gateway needs some work.

Yeah, I tend to agree, I just didn't want to be a total jerkoff on the thread.

250ms we can live with for many things, but not 5x.

But yes - it should be under 50ms at all times I think for people to start taking it seriously.

The 5 second cold request thing is what GAE had 5 years ago. Eventually they gave up trying to make startup faster and allowed a paid plan to stay always hot. Which was good, but then also moving you much closer to a managed server (vs a pure pay as you use thing)

No way you're getting 15 ms on a TLS connection over the internet. I've set up similar pings with API Gateway and API Gateway -> Lambda, and it seems with the APIGW the TLS connection is the most expensive part. On an established connection I've seen 10 ms overhead which is not great but worth the value in some cases.

If you are serving high traffic then what's the problem?

If your product has only occasional use then you trade cost for latency; spin up an ec2 instance if your request times exceed a few minutes call

You could have certain requests that don't come that often, even on a high traffic site. And if you don't know for how long lambdas stay hot, how are you supposed to time it? Even a call every few minutes may not be enough to keep them hot.

Ah, I see. Perhaps that would be a reason to keep an ec2 instance spun up to host the functionality that must always have latency < warmup+runtime? Surely suboptimal. This makes me consider running a cron job to poke timing critical lambda workflows every few minutes...which to me imply writing my functions in such away as to expect this "heartbeat" and act accordingly...and that just feels plain icky...

Reply on news.ycombinator.com