Hacker News Reader

As far as “bottlenecks”.

Almost all applications have data separate from code. Most implementations don’t run their applications on the same server as their databases.

Most app servers shouldn’t be “caching state locally” anyway. Once you do that it makes scaling horizontally harder - you have to Implement sticky sessions.

And in all practicality, I would assume that most people are using EBS for their storage which is network attached storage.

Two lambda backed microservices work together just like any other microservices - via known urls and over https.

>you have to Implement sticky sessions.

There is more to the world than serving HTTP requests.

To expand on your point, there is also more to cache than user-specific data. Exchange rates, tax rates, product catalogs, and more can be usefully cached without requiring sticky sessions.

who keeps that data locally on the server?

If you're querying the IRS for tax rates every time you run a calculation, you're doing it wrong.

If you aren’t querying exchange rates often - you’re also doing it wrong....

Depends on what you're using the exchange rates for. The rates used by Visa only change once per day, as an example.

And you still wouldn’t store that locally. You may cache it locally - something you could also practically do with lambda.

Well yes, the parent we're replying to was talking about caching.

Anyone implementing business-critical applications which have to continue to function if external services go down.

Ever wondered how cash registers work? Exactly like that. The world may burn down around you, but as long as your cash register PC is running, it must continue to at least offer basic functionality.

You mean like all of the cash registers at companies like Target that were breached because they were distributed?

https://www.post-gazette.com/business/2013/12/19/Target-ackn...

The vast majority of point of sale systems are networked, and this is nothing new. There's literally millions of PoS systems out there running off of AS/400 backends, even today.

Things like exchange rates, catalogues, tax rates are not generally 'session oriented' anyhow. They could be cached by some other means hopefully at least somewhat transparent to the business logic anyhow, so to the OP's point, I don't think this is a limitation.

I'm in agreement that the fundamental limitations prescribed in the article are academic for the most part. 'custom GPU's ' are not really what we're after with Lambda's etc, at least not yet - and I'll bet when we get to that point it may be possible to access specialized hardware in Lambdas as well.

In addition, things like '15 minute timeouts' are kind of arbitrary product design issues: we generally don't need 'long-timed Lamdas' because that's not what they are for. Something that needs 15 mins of crunching is better suited to another piece of the infrastructure anyhow.

Serverless computing, done in a manner that can doesn't throw up a bunch of hurdles definitely has a lot of advantages.

The more obvious downside to me is the degree to which they are all fairly custom implementations, and that Lamdas aren't so easily ported between providers.

Sure, you have communications over queues, event messages etc. both supported by lambda. You also have higher volume messages from something like IOT devices where you would use something like Kinesis.

Are FaaS’s fundamentally aimed at those use cases?

Yes. Those are all processes that are “embarrassingly parallel” workloads. You can just point lambda to a queue for instance and it will automatically scale based on workload. When nothing is in the queue, you’re not paying for resources just to poll an empty queue.

>Almost all applications have data and separate from code.

Almost all applications aren't distributed in any meaningful sense of the word.

If you aren’t doing distributed applications, why are you even considering lambda - the main purpose of which is massive parallelism?

Because 'you' don't get to make the decisions.

What type of organization will ignore experienced developers that can rationally argue the type of architecture? I've been developing professionally for over 20 years and I can honestly say that I’ve never been overridden about small scale architectural decisions like should we use lambda versus an EC2 instance unless the decisions makers had some larger insight than I had.

I don’t mean I could have decided whether to go to Azure or AWS or use Oracle instead of SQL Server or something on that level.

I'm not in this arena at all but Amazon's lambdas seem like a way bigger architectural decision than azure vs AWS or Oracle vs sqls

It’s acrually smaller. A lambda code wise is just a function that takes in two parameters. Like I said before, architecturally, your lambda entry point should be skinny - take in lambda event data, map it to your domain object and call your domain services (not http services, your business layer).

In fact, you can take a standard C# web api and using the AWS SDK deploy the same code either as a lambda or you can do a standard deployment to IIS or a self hosted executable with Kestral

I’m speaking in terms of .Net but the same concept applies to any language.

One project has your lambda interface with a dependency on your domain classes.

A second project has your standard controller code with a dependency on your same domain classes.

You can have yet a third project that in my case uses TopShelf - a nuget package that makes it easy to create Windows services - that also uses Kestral to self host a web service.

I can deploy the same code to either IIS, Windows as a service, a Linux VM (with Kestral without TopShelf) or lambda. The only difference is the code deployment part of my CI/CD Pipeline. I use AWS CodeBuild with a preconfigured .Net Core Linux Docker container that builds to either target and stores the artifact as a zip file to S3.

None of this onerous. I test locally on my Windows machine and just push to git everything else just works based on some yaml files.

For us, lambda is great for the random junk drawer of compute load. Cron jobs; miscellaneous async jobs we'd have used something like sidekiq for in the past; infrequently used microsevices (where cold start doesn't matter) like schema registries (where client caching means it might not be hit for days). In the past I've had "job servers" running these various things.

Yeah, the article isn't arguing about those things, but the rest of the jobs, especially data dependent things. For non time critical miscellaneous stuff lambda is great.

Is the main purpose massive parallelism? I was under the impression it was mostly for rarely used resources, because you only pay when you use them.

I think another very huge benefit is that developers don't have to worry about infrastructure.

I don't know if its there yet but at least I imagine that's something serverless technologies strive towards

Not worrying about infrastructure implies trivial portability between IaaS providers... Is that at all true?

Again in the real world, an AWS shop isn’t just going to pick up and decide willy nilly to pick up and go Azure just to temporarily save a few dollars.

> Most app servers shouldn’t be “caching state locally” anyway. Once you do that it makes scaling horizontally harder - you have to Implement sticky sessions.

Why not? In-memory caches are fast and fairly uncomplicated to write.

The sticky session thing is about a clients session. You store the identifier on one machines and if the same client hits another machine it knows nothing of that session I'd. This is one example but in memory only works well in this setup for client agnostic data. Type table lookups, etc

Because when you have horizontally scaled to 1000 servers, when the next request comes in, it's very likely to hit a different server from last time.

So your cache hit rate is going to be horrendous. Take the cache hit rate you'd have for a single server, and divide it by 1000, the odds of getting the same server again. So if your cache hit rate was 80%, now it's 0.08%.

Of course you could arrange for a user's requests to hit the same server as last time, but that's sticky sessions.

Wait so how is Lambda making this worse? In this case it’s pretty clear you want a caching server instead of an in memory cache. I don’t see why the 1000 serverless functions is worse than 1000 long running processes.

That’s kind of the point. People act like lambda is some weird thing that completely changes your architecture.

Even when I’m hosting things on VMs, I still consider them as disposable. I don’t store state on them, logs are all sent to a central store like Cloudwatch or ElasticSearch, if a health check fails, autoscaling just kills the instance and initiates another one based on a custom prebaked image and a startup script, etc.

Long lived data is stored in a database and cached with either Memcache or Redis.

We had an old school Devops guy that wanted me to list the names and ip addresses of some servers that I was using. I told him that I would have no idea and even if I did give him a list, it would change tomorrow. I have some windows servers that in non prod environments that host some Windows services based on queues. The servers are automatically terminated when nothing is in the queue and it may spin up a dozen when needed. My servers don’t have names - they have tags.

How is this any different than using a Lambda?

> Hiw is this any different from Lamba?

You had to write all that orchestration code yourself.

That said, I completely agree with you, and utilize the same arch at dayjob.

There are “app tier” cattle, and there are a few special pets (databases) that are backed up, loved, hand-held, etc. because database failover is still painful for users for seconds-to-minutes. When you’re serving thousands of requests per second, that’s a lot of angry users.

RDS, Azure SQL, Cassandra, Aurora, whatever still cause measurable pain for the end user during node failure.

You had to write all that orchestration code yourself.

With the Windows VMs I had to:

- integrate TopShelf to make it a Windows Service

- write the code to poll the queue (no big deal but still)

- create two Cloudwatch alarms to monitor the queue to tell the autoscaling group when to scale in and out

- define the launch configuration for the autoscsling group

- define the autoscaling group

- add health check functionality to the Windows service to ensure that the Windows service was running that could be integrated with the sutoscaling group (well I didn’t have too but to be complete)

Alternatively with lambda all I had to do was

- add a method that took in two parameters - the SQS event and the lambda context.

- add the SQS event to trigger the lambda.

Of course both stacks were created with Cloudformstion. One I had to do myself, the other I just basically set everything up in the web console, exported the CloudFormation template and made some minor changes.

You always have to apply pressure to keep the system decoupled or you get a Ball of Mud. If you switch to micro services and serverless and then have a high fanout and caches everywhere just to make the numbers work, you haven’t gotten rid of the Ball of Mud, you’ve just hidden it in plain sight.

Serverless and embarrassingly parallel problems look like pure functions punctuated by one or two state changes. To get that you should probably be passing all the data in, instead of communicating out of hand with a cache to keep your fanout from looking awful (again, hiding the problem).

But who is doing the "passing-in" then, and where does the data come from? This still sounds like hiding the problem...or more specifically, making it someone else's problem.

Usually the data comes from outside the system - an api integration, user interactions with a website, an ETL process involving a file, IOT devices, etc.

Who does the passing in in development mode? Where does the data come from?

You have this problem anyway, and it blocks you from launching new code. These out of band communication mechanisms always lead to problems with developer ergonomics and repeatability.

It's essentially a larger version of the problem of testing pure functions versus heavy usage of mocks. Passing the data in directly makes testing the callee very simple.

Not every application is a web application. There is this quaint old framework for distributed computing called Hadoop, maybe you've heard of it? It has a little brother Spark. Anyway the whole point is to "ship code to data". Its an interesting idea, but I don't think it will ever go anywhere.

And surprisingly that use case works with Kinesis and queues etc.

No, map reduce is not stream processing at all. It’s a transformation/computation over what may be many terabytes of data.

https://aws.amazon.com/blogs/compute/ad-hoc-big-data-process...

Yes its possible, and the SP describes the shortcomings involved in giving up data locality and cross-node interaction.

> there is no network addressability of serverless functions

Didn't inetd (4.3BSD, June 1986) solve the problem?

You're not wrong.

There is nothing stopping inetd services from sharing disk data or communicating with other local services.

The particular inetd was just an example, but even for use with exact inetd from 1986, there are many solutions in modern operating systems to restrict any process, like SELinux, AppArmor, Seccomp and capabilities, LXC unprivileged containers and whatnot.

No. It solved a problem that was vaguely analogous if you consider a binary sitting on some Unix server the equivalent of a “serverless function”. They are somewhat similar, in that both are relatively self-contained and stateless compared to the alternative, not requiring anything to be explicitly spun up or down (a daemon or a VM/container respectively). But an inetd binary runs on an existing server, and the client is expected to know the address of the server. A “serverless function” can be spun up on the fly on any number of provider-owned servers which nobody has seen before, so there needs to be a separate mechanism to (1) request a server to spin up if necessary and (2) locate that server. Right now, Amazon Lambda only provides this through high-level abstractions such as Amazon API Gateway; you can’t even listen on a port from a Lambda container, so you’re limited by the speed of those gateways.

"Locate that server" is DNS. Or possibly reverse proxying, if DNS is hard and Apache is easy.

Transparent elasticity is the fundamental advance here, and it seems to come with a surprising number of tradeoffs.

Regarding the bottlenecks, note that CSPs have been doing work to run functions next to the data without having to “retrieve” it to a warmer storage level.

Consider Amazon Web Services’ S3 Select, Glacier Select, and Athena introduced a year ago:

https://aws.amazon.com/athena/

If you look at 2017 announcements then 2018 announcements, you can project a likely strategy here.

Very interesting. I wasn’t aware that with S3 select [0] you can select columns by using SQL on a zipped file stored in s3.

[0] https://aws.amazon.com/blogs/aws/s3-glacier-select/

That's not a disclosure, that's an ad.

So knowing this what has Gitlab done to overcome it?

If they haven't, why not? Why is it hard or inconvenient?

What we now shipped is a Minimal Viable Change (MVC). I suspected that serverless wasn’t ideal for all use-cases but I couldn’t articulate this until I read this paper.

We have taken no steps to overcome these problems, I think we’re likely to follow the leaders in the industry like Knative from Google and TriggerMesh.

In general I think serverless is more of a pattern then a technology. We’ll use Knative to offer PaaS functionality (Heroku) on Kubernetes with GitLab.

I find it kinda funny that serverless can't do GPUs while docker containers mounted with the nividia-docker runtime can do GPU capable tasks.

Are these serverless solutions really so different?

Yes, they usually have greater levels of isolation.

Serverless/FaaS is just a buzzword which describes proprietary hidden abstraction on top of vendor's isolation system (VM), proprietary image and orchestration APIs. One day they may happily announce that they allow you to:

   @Use(GPU)
   func yay(...) {
   }

I'm interested to pick your brain about the second fundamental bottleneck you/the article mentioned. I'm not sure that I agree that serverless functions are fundamentally non-addressable. Today they're not addressable, but why can't they be in some future evolution of serverless? Sure serverless functions are ephemeral, but during their brief lifetime, is there something fundamental keeping them from being addressable?

I’m not a serverless expert. Consider opening an issue with your proposal, or even better a merge request against the serverless functionality in GitLab that is already in master.

Implementation details, not conceptual ones