Scaling to 15000 Functions and Beyond
openfaas.com
openfaas.com
You can lazy load your route implementations
I've never attempted lazy loading inside a Faas platform but I think generally you would be better off creating small deployable chunks and letting the platform handle scaling them up / reusing as necessary. Less fighting against the platform and if you have to maintain strict boundaries to enable lazy loading only the needed code for each route it's a short step to deploying those as their own functions anyway.
I don't know what PaaS you've used, but at least with Google Cloud Functions, you can do a minimum number of instances. Set it to 1 and you never have cold start issues.
There is a middle ground that will be much faster and cheaper all around.
It all kind of works while the fanout is low, but these systems have such long feedback loops that the team doesn't really learn how to avoid the degenerate cases and the whole thing limps along. The system isn't just allowing emergent behavior, it depends on it. Which is a special kind of insanity I've never been comfortable around.
Really, just write some code that does exactly what it says it does. It seems remedial at first but once you get going you start finding ways to add features that were difficult before and it all comes out in the wash.
The OP has also hacked away on OpenFaas in the open for years, giving us a portable alternative to lambda. That seems to have a pretty big value prop considering how lambda ends up being fairly expensive at scale.
Though I don’t know their motivation for hiding pricing, I suspect every enterprise level deployment (for instance into an ISP) is pretty bespoke.
and “bespoke” pricing for such models is also pretty common. doesn’t mean we have to like it.
> That seems to have a pretty big value prop considering how lambda ends up being fairly expensive at scale.
Portable is valuable, yes, but at what price ? Does anyone know what the price is ? If it’s customer-specific it’s impossible to say that this proposition is generally competitive with lambda.
Open core / commercially support products need to have published pricing for at least a standardised offering IMHO. If I need to get on a call with a salesperson to find that out (so I can do the back-of-the-envelope calculation that says whether or not this thing will cost in) it isn’t happening…
A free, MIT license is limited to 30 functions. That's enough to basically play with it, but you probably can't do anything serious. That leaves purchasing a license mandatory, but if you're pre-revenue, boot-strapping a business, that is impractical, if not impossible. Thus you pretty much have to go the vendor-locked-in lambda way; even if you already have the hardware (and expertise) to run this thing.
It is worth noting that azure functions can be run self-hosted, entirely for free.
I don't understand: if they used MIT license, what's to stop me from increasing this purely arbitrary limit?
Personally I've never had a use case for OpenFaaS but if I did, it would probably be one of the first things after installing.
So, yeah. It’d be an interesting case, but hopefully I can just eat popcorn on the sidelines. I don’t want to be the one fighting that battle.
And even then you better have the configuration defined in code because otherwise you'll spend an entire afternoon reverse engineering how it works.
I believe there is very early stage work [0] in this direction, but obviously nothing going mainstream yet. Would love to find more examples in the wild if anyone knows any!
[0]: https://egghead.io/blog/how-to-quickly-build-a-backend-with-...
The example also mentions Stripe payments, but nowhere does it mention transactions or error recovery because they're stupidly difficult in microservices designs, so we'll just "skip right over that" and go straight to losing money any time there's a hiccup.
"Sure it's incomplete and outright dangerous to our bottom line, but I can't wait to tinker with it!" is precisely the problem with the industry.
Now if a runtime could reap the benefits of this architecture automatically, sort of like a compiler would optimize code for a particular cpu, then that would be a best-of-both-worlds situation.
The industry has had plenty of opportunities to ditch them and go back to monolith services and yet it hasn’t happened. Is it because everyone is wrong except for you ? Or is it because they are better for certain types of use cases.
https://www.erlang.org/doc/design_principles/des_princ
WhatsApp famously scaled a lot with relatively few devs:
> WhatsApp scaled to 1B users with only 50 engineers https://news.ycombinator.com/item?id=28985169
And yeah, Elixir is great!
The language is built with this idea from the ground up, so it can move functions around between machines robustly and without running into problems with versioning. It's the only model I've seen for distributed computing that really makes it a first-class citizen of the language.
It’s not wise to dismiss the technical decisions made by developers as “playing with the latest toys”. Because you are ignorant. You have no understanding of the business requirements, NFRs etc that have led to the decision. So I prefer to give developers the benefit of the doubt.
Lambda architectures have trade-offs like every architecture and they are useful in certain situations. Building everything in one service and scaling it when the components have different scaling behaviours usually isn’t ideal.
Address this part, or you’re just deflecting. Comments? Suggestions?
And they suit use cases where they are independent from each other and can be tested in isolation eg transcoding.
In the early days of AWS Lambda I saw teams trying to apply older techniques to lambdas and get stuck with things like distributed transactions but industry has learnt a lot since then. So as always right tool for the right job.
Trading developer experience for just about anything does not scale - up or down.
It’s a false economy.
Writing some code and throwing it over the fence to some Ops person to manage is not workable long term. It’s why DevOps exist because developers have to take some responsibility for operating their own code. And Lambda architectures help with this.
Statements made confidently while also being totally untrue.
Hacker news is like that, the person you reply to could be between the person that invented and built the tech discussed or a kid replying quickly before going to do their chores. Always fun to see either way
> If it doesn’t work for the developers, then eventually it won’t work for anybody.
Don’t conflate technical challenge with logistical challenge. Picking between two algorithms for condensing data is technically challenging. Picking between a solution that is transparent and one that is opaque are not the same kind of challenging. As an old mentor used to say, “I hate the term ‘tech debt’. This [situation] isn’t tech debt, it’s wear and tear.”
Opaque is magic. Hiring people who understand it is expensive. Building people who understand it is part luck and part patience. And the opportunity costs, which we almost never account for, let alone properly, are one of the biggest consequences. What else could you do with that time and, more importantly, energy?
As far as I know, Google does not actually use k8s for their public facing services (or anything at all?). I would love to be corrected on this.
Google was able to scale quickly back in the day by keeping things as simple as possible. "1 hard drive per server" level simple. Center everything around one massive job scheduler (Borg).
Of course a lot has changed since that time, but I can't see Google upending a established infrastructure by making k8s plugins for their network requirements, tying k8s into Borg, etc. It would give less visibility and debugability for little gain.
(But again I have zero inside knowledge here)
At scale there is no one size fits all. Kubernetes can fit a lot, but will always fall short of a tailor made solution.
For more unsolicited advice from an armchair infrastructure hyperscale expert, please subscribe to my newsletter.
* 0..N Scaling
* Request/Invocation is unit of compute
* Deploy anywhere
* Don't need to bring a web server
* Instant Observability (distributed tracing, metrics, logs, alerts, & dashboards)
* Instant Security (AuthN/AuthZ/mTLS)
* Instant Cost Monitoring
How good "FaaS" is will be highly dependent on how good your ops team is.
From my own personal experience, Google Cloud Functions, and everything around it, is fantastic. Had 25k "worker" computers hitting GCF 24/7 (~50 rps) reporting their status. New status message went onto Cloud Tasks, which just hits GCF again, and stored into a Cloud SQL Postgres. Cost was about $200/mo. It ran perfectly with zero downtime for over a year.
Super easy to develop, test, and deployments were done with github actions. Very little actual code is dependent on anything GC, so vendor lockin was minimal.