> Most app servers shouldn’t be “caching state locally” anyway. Once you do that it makes scaling horizontally harder - you have to Implement sticky sessions.
Why not? In-memory caches are fast and fairly uncomplicated to write.
Why not? In-memory caches are fast and fairly uncomplicated to write.
Serverless and embarrassingly parallel problems look like pure functions punctuated by one or two state changes. To get that you should probably be passing all the data in, instead of communicating out of hand with a cache to keep your fanout from looking awful (again, hiding the problem).
You have this problem anyway, and it blocks you from launching new code. These out of band communication mechanisms always lead to problems with developer ergonomics and repeatability.
It's essentially a larger version of the problem of testing pure functions versus heavy usage of mocks. Passing the data in directly makes testing the callee very simple.
So your cache hit rate is going to be horrendous. Take the cache hit rate you'd have for a single server, and divide it by 1000, the odds of getting the same server again. So if your cache hit rate was 80%, now it's 0.08%.
Of course you could arrange for a user's requests to hit the same server as last time, but that's sticky sessions.
Even when I’m hosting things on VMs, I still consider them as disposable. I don’t store state on them, logs are all sent to a central store like Cloudwatch or ElasticSearch, if a health check fails, autoscaling just kills the instance and initiates another one based on a custom prebaked image and a startup script, etc.
Long lived data is stored in a database and cached with either Memcache or Redis.
We had an old school Devops guy that wanted me to list the names and ip addresses of some servers that I was using. I told him that I would have no idea and even if I did give him a list, it would change tomorrow. I have some windows servers that in non prod environments that host some Windows services based on queues. The servers are automatically terminated when nothing is in the queue and it may spin up a dozen when needed. My servers don’t have names - they have tags.
How is this any different than using a Lambda?
You had to write all that orchestration code yourself.
That said, I completely agree with you, and utilize the same arch at dayjob.
There are “app tier” cattle, and there are a few special pets (databases) that are backed up, loved, hand-held, etc. because database failover is still painful for users for seconds-to-minutes. When you’re serving thousands of requests per second, that’s a lot of angry users.
RDS, Azure SQL, Cassandra, Aurora, whatever still cause measurable pain for the end user during node failure.
With the Windows VMs I had to:
- integrate TopShelf to make it a Windows Service
- write the code to poll the queue (no big deal but still)
- create two Cloudwatch alarms to monitor the queue to tell the autoscaling group when to scale in and out
- define the launch configuration for the autoscsling group
- define the autoscaling group
- add health check functionality to the Windows service to ensure that the Windows service was running that could be integrated with the sutoscaling group (well I didn’t have too but to be complete)
Alternatively with lambda all I had to do was
- add a method that took in two parameters - the SQS event and the lambda context.
- add the SQS event to trigger the lambda.
Of course both stacks were created with Cloudformstion. One I had to do myself, the other I just basically set everything up in the web console, exported the CloudFormation template and made some minor changes.