I've read a lot of these style posts, and oftentimes the results don't end up being that interesting - but I have to say this post is different. Nice job and good overview of all the different layers in the stack. I didn't realize there was a meaningful difference between Cloudflare Workers and Lambda, but now will have to check it out.
I don't understand this.
The entire point of Lambda's is to offload the overhead and complexity of managing severs - and - to allow for large occasional spikes in traffic.
Even with the limitation of '5 second warmup per concurrent' Lambda - that's not a big deal. It means maybe 100-200 users have to wait a few seconds while 100-200 Lambdas warm up - and with 1.5 user spike ... you really don't know what you're going to get, but with Lambda's at least you're pretty much guaranteed it will work.
For cost efficiency, with stable tech, stable/predictable traffic, and enough scale that you have the right team to be able to manage your EC2's properly - yes that makes sense. But you need some scale to get to that point wherein that cost efficiency is worth it given that Lambda's 'just work' fairly well fairly easily.
But I definitely could be missing something.
https://docs.aws.amazon.com/lambda/latest/dg/invocation-scal...
Based on the scale mentioned in the article (hundreds of RPS) it's likely Lambda would also have been able to handle it just fine.
On another note, using non-provisioned infrastructure (aka "serverless") for an expected bursty load (TV campaign) is bordering on negligence. It sounds like lots of potential donations were missed because the Stripe account was not set up to cope with the load. It turned out to be a wash because Stripe donated 100k but if you change the business context of this system from "receiving donations" to "taking credit card payments" ... this outcome would not be considered acceptable.
But why is running a bursty tv load on serverless negligent?
"the Stripe account was not set up to cope with the load." - I've used Stripe and I don't know what this means. I don't see any 'product' limitations towards accepting many payments, that said, they could have some internal financial controls which should have been accommodated by calling ahead and letting them know about the burst.
And how would there be a technical limit with Stripe? If payment processing was directed to Stripe.com - surely they can handle the traffic.
Because Lambda and any other FaaS platform are always going to have some limitations as to how fast they can scale out. When you put stuff on TV, this is as bursty load as it gets. You are literally telling everyone viewing that TV channel at that time to pick up their phone and browse to some link.
If you are expected to handle all this load, the only way to ensure you can handle it is to over provision. The FaaS may promise some burst rate, but ultimately it's a shared hosting platform and if they have to choose between giving you your 3000 containers or keeping the other X customers running on those same boxes healthy they will always opt for the latter.
Worst case for them, they refund you the few bucks you paid for Lambda during that time. Worst case for you is you miss a ton of payments/donations/whatever. Welcome to the cloud.
The Stripe API has rate limits of 100/sec: https://stripe.com/docs/rate-limits
They put that to protect themselves from getting overloaded and Im sure these limits are more than plenty for typical use of Stripe which would be well spread out throughout the day. But when you put something on TV, 100 RPS is no way near going to be enough.
Stripe: 100/second seems like perfectly ample headroom to support their payments. At $10/payment, that's $1K/s which is $3.6 Million in one hour.
If that's 'The Stripe Limit' that every Stripe customer has, wouldn't it kind of imply that's plenty enough for even much bigger customers?
As 'FaaS' scaling - as far as I knew that was the entire point of Lambdas - that they could scale quite quickly.
The alternative, EC2s would need to also 'scale very quickly' and could very well run into the same 'prioritization' problems, no?
In reality, I don't think there's a problem here with Lambda - while possibly not the most ideal option - I think that AWS has ample overhead to supply this little company with their little burst.
Amazon accommodates some pretty big workloads. A call to AWS support may very well have supplied them with the answer as to the real limits on scale.
Also, Workers supports HTTP3 (QUIC) and ESNI already.