If anything mail is pretty much the easiest thing you can possibly pick to scale, because the inbound mail will be automatically retried. And haproxy in front of SMTP servers works just fine (really, any load balancer that can load balance raw TCP connections, but I've used haproxy to load balance thousands of messages a second).
For your user-facing side you need a bit more resiliency, but nothing stops you from using a service like SES to back a traditional setup for sending either. Reliably scaling outbound mail is the easy bit - the hard part is the reputation management that managed mail services provides, and no specific deployment mechanism will solve that.
^this. if you want to send email its not hard....but if you want that mail to pass spam filters its a different problem altogether. hosted services like SES and mailgun will expose problems in how you are using emails (not handling bounces, not handling unsubscribes, etc) and in our case was very helpful.
It may seem like a nitpick or something not worth worrying about, and for most that's probably the case. But for some businesses it could be a crucial difference. My point is simply that this is a legitimate benefit of serverless that wasn't mentioned above--I didn't think that would be a controversial point.
The precise extent is different between different platforms depending on overheads, but that just means the point at which you need to trigger scaling is different.
You can find lots of descriptions of approaches people have taken to keep serverless instances running to avoid the cold start delays to work around this... For autoscaling groups you'd instead configure the alarm points used to trigger the scaling accordingly.
Serverless platforms tends to assume the startup will be fast enough to keep the connection open rather than return an error, but that is a load balancer config issue - you can set up retries and wait for other platforms too if it makes sense.
(Though for email this really does not matter - retries for all commonly used mail servers follow some form of exponential backoff up to many hours at least; retries works just fine in practice)
Serverless deployments are just another ladder step up the abstraction level, continuing the tradition that hardware doesn't matter. Similar to code compiled into assembly or a garbage collector managing memory. In the common cases, these cases are harmless (otherwise they wouldn't be popular), but they generally hide what's actually happening. Doing a garbage collection on a 200MB app is generally pretty snappy. But doing one on a 32GB server app can take seconds or minutes.
Abstractions like these are fine, as long as the limits of the abstraction are well understood. Sadly, that is rarely the case.
The high load can be alleviated by the use of more MX server DNS records (and the MX servers of course, across the different locations), LBs, smarter thresholds. Of course nothing is a panacea.
Either way you will hit the AWS's limits or will get a huge bill. And then, even if you set up the budget limits, it still won't make the service more available once you reach the limits.
But apart from that, there's a huge benefit in saying "I'm happy to spend any amount up to X" and not needing to do any capacity planning beyond that vs. continually trying to guess what's the right % to over-provision your VMs and having downtime when you get it wrong.
> If you're running a saas and the increased traffic comes from paying customers, you likely prefer a huge bill to downtime.
Well, in such situation, I would probably run more advanced container orchestrators such as Kubernetes which you will then configure to automatically spawn the additional server instances.
Of course there are certain advantages in running a serverless code as you have just mentioned, but since my primary concerns are "my data is mine" + "no vendor lock-in" + "I control all the gears", it is not the best option for me. Unless I want to run & provide the serverless services by & for myself.
It's always a game between the security (more freedom) and the convenience (less freedom). Though, for some, there is more freedom in the convenience (until they start to see their hands are in the digital cuffs :P)
Auto-scaling helps, but it still takes awhile to spin up new VMs, and you'll have downtime in the meantime for sufficiently large bursts.
On lock-in, in my experience with any non-trivial infrastructure you end up tied to your provider anyway. You're still going to need IAM users and VPCs and subnets and provider-specific auto-scaling and all that jazz. Serverless lock-in is deeper, but either way switching is a big project.