Lambda Store: Serverless Redis
medium.com
medium.com
For start, you want redis to be as near as you can to your application. It many times is used as cache and it makes no sense to have long latencies to your cache layer (many times Redis is even deployed in the same pod as the app using it precisely because you want it to be nearby). And if your infrastructure is already in AWS (why would you choose elasticache otherwise?) you would be paying all the data from AWS to your external Redis as a service provider and that might be much more than what you even expected to save in the first place.
To be honest, AWS elasticache is not even an expensive service (t3.micro instances work just great and allow you do no upfront reservations, which you failed to use as comparison for obvious reasons).
Really, I don't think Redis is a problem to solve and I'd put my money on someone giving cheaper documentdb alternatives, or redshifts, or managed ClickHouse services, etc. Those are the real killers!
Anyhow, sorry for being a bummer and wish you best of luck!!!
However, as someone else also mentioned, we need alternatives to other AWS services, those that are expensive enough to run them on prem. The day you offer cheap Elasticsearch, ClickHouse, documentdb, etc, I'll kill my Hetzner machines and will come to you, sir.
The real pain with using both is the VPC requirement.
I'm actually hoping something like lambda.store can remove the EC2 NAT gateway overhead.
This is not a selling point to me. When it comes to storage, I prefer experience over motivation, just like I prefer durability over speed (cough, MongoDB).
Otherwise though this looks pretty cool.
I do wonder what their scaling story looks like. How can they maintain a profit at such low prices and handle sudden load spikes?
We are not using shared Redis instances. We have our own Redis server implementation, which has a tiered storage model which keeps the hot entries in the memory to better utilize memory usage. Using this we can use smaller instances for the most of the time and migrate the Redis db to a bigger machine in a few seconds when needed.
We are not using Redis OSS, instead we implemented our own Redis server. Because it's not very possible to scale it without increasing the costs and also it would be very challenging to adapt serverless use cases.
Different from the Redis OSS, we are using a tiered storage model which keeps the hot entries in the memory and gradually offloads colder ones. Also we can migrate a Redis DB to a bigger machine in a few seconds when a database needs more resources. We will share more technical information about our architecture in Lambda Store blog.
but this sounds even worse to me. okay so you have API compat great, but now you have unproved code running under the hood wonderful.
Currently, looks like the current version is limited few AWS regions. This may not be a good fit for those outside the AWS ecosystem. The main reason users use Cache Stores is to remove latencies of hitting a database or re-computations. This may not work out for those on DigitalOcean or Azure endpoints.
Also, within the AWS ecosystem, power users may not use it since it bypasses IAM and no insight in case of failures because of connection limits, as well as no SSL in the free version.
A note about "accessibility" on the page, table data is represented as an image and screenshots from Twitter should be have embeds. Otherwise, all users cannot read the content.
I would like to use something like this but with no SOC2 or any other security certification, storing customer's data out of your cloud provider is a no-no for me. On top of that, there are no ACLs, nor any fine-grained control access. No audit log, SSO, or similar features are required.
So I wonder what market is this trying to capture? If you're not happy with DynamoDB and you need even better latency, it means that you're running this in production and probably handling sensitive data. Would this service be viable for you? Just curious what the HN crowd thinks.
EDIT: Comment bellow me said this:
> TOS says not to upload data that "contains personally sensitive information" so that may limit some use cases
That, for me, concludes this is aimed at casual use.
Sure, we are already planning to post blogs about our architecture details. You can follow us on medium: https://medium.com/lambda-store
https://docs.lambda.store/docs/overall/rediscompatibility/
From a customer's perspective, "serverless" means consumption-based pricing as opposed to reservation-based, regardless of the actual technology being used.
Classic Aurora for instance is not “stateless”. You still have to appropriately size the underlying server, you pay for it whether you are using it or not, you might need to reboot it, etc.
DynamoDB on the other hand is considered Serverless by AWS because you don’t size the “DynamoDB server”, and it can scale write and read capacity automatically.
Willing to be proved wrong, but what I've seen so far does equate serverless with stateless, at least at the compute tier. Aurora has no "Lambda" or "serverless" branding, right? Any stateful stuff is sort of called out as an integration.
And there is Serverless Aurora
https://aws.amazon.com/rds/aurora/serverless/
Also here there are many stareful serverless products:
https://aws.amazon.com/serverless/
It’s more about compute than storage.
You can now have stateful storage attached to AWS Fargate - serverless Docker. By attaching your containers to EFS.
https://www.sdxcentral.com/articles/news/aws-adds-direct-sto...
With regular Aurora, whether you are using it or not, you’re always paying for both the server and the storage and you have to provision the server for peak workload. With Aurora Serverless, if you don’t connect to the database, you only pay for storage.
With lambda and to a lesser extent Fargate, you don’t pay for the underlying server at all until you actually need to run something and then with lambda you can scale up to as many instances as your account allows (a soft limit you can ask for more anytime) and pay nothing when you don’t.
With EC2 you have a server sitting there listening for events whether or not anything is sending you an event.
I'm going to give it a go! The only thing that jumped out at me, though, was in the TOS which says not to upload data that "contains personally sensitive information" so that may limit some use cases.
Update: I've created a basic free database for now. Using it with redis-cli with no real issues so far but not stressed it yet.
The thing is, however, as someone else touched on, the pricing of Redis Labs is still reasonable, and despite feeling outdated, it's also stable and a safer bet... So I don't really know how many organizations are willing to trade cost-saving/coolness for higher risk, at least when it's still new and not well established.
How do you plan to address this type of concerns?
When you have steady traffic, it makes more sense to move to reserved pricing. Currently the reserved pricing plans start from 500 req/sec. We may have to have more plans to cover smaller throughput cases.
Just my 5c.
Whereas there is such a thing as fully peer-to-peer systems without actual servers. It's almost as if the people who coined and marketed the term want to consolidate the idea that servers/backends are an inherent property of information systems.
It should be said that I am a both a believer in that the words we use shape they way we conceptualize and reason about things, and also a proponent for less infrastructure centralization.
Now you might say, "we already have the word peer-to-peer with that meaning". IMO peer-to-peer has been diluted to the point of being practically meaningless, commonly used to describe systems such as Google Hangouts.
I don't believe there is any kind of conspiracy or conscious effort on the side of vendors or providers to do this, but effectively we don't have a word to describe fully decentralized/distributed/peer-to-peer/serverless software today because all those words have either been diluted or have a different meaning. It gives me some 1984 doublespeak vibes.
Besides all the above: My original comment was meant as honest advice that they should reconsider using the term this way, given both that:
* I'm not the only one who feel this way, so it will likely put off other potential customers as well * There is disagreement on what the term should mean even as commonly used (just look at other comments here arguing that nothing with state between invocations can be considered "serverless")
I think the intended meaning will come across much clearer by instead calling it "Fully managed Redis" or, if it's crucial to get across the billing model, "Fully managed, pay-as-you go Redis".
Since we are not using OSS Redis code, it would be very difficult to adapt it to the serverless model, we cannot support Redis modules directly. But after completing missing Redis commands, we can work on modules support too.