Hacker News Reader

Comment by boynamedsue | Hacker News Reader

Aurora Serverless is an absolute nightmare to use from the perspective of Lambda. It’s easy to flood the database with Lambda connections and the database layer doesn’t scale automatically as described. So the result is that Aurora Serverless returns max connections errors.

The RDS Proxy isn’t GA yet and has huge disclaimers on using it in production.

The only alternative is to use the RDS Data API which requires sending raw SQL queries over HTTP.

If you’re using an ORM getting the actual bound SQL query from the database request can be a bear.

And to make matters worse the RDS Data API doesn’t return JSON results with the tables column names. It puts the column names in a separate key that requires developers to map to generic column names using the index of the value.

AWS should be embarrassed about Aurora Serverless. No question in my mind.

View on HN

You didn't even mention the hard 1MB response limit with the RDS Data API.

>> ... database layer doesn’t scale automatically as described.

Can you give more details on what you mean by this?

>> RDS Proxy

TIL about RDS Proxy. Looks interesting for newer projects, hoping that it becomes GA in the coming months.

All "elastically scalable / autoscaling" cloud services (e.g. S3, ELB/ALB, Aurora Serverless, etc.) are ultimately composed of application code running on a finite number of machines thereby capable of serving a finite throughput of load. Excess load must be temporarily shed while additional machines are spun up. You can easily observe this behavior by running load tests on these services.

Separately, all applications don't scale linearly with the volume of open connections. There's typically a sweet spot of open connections that provides maximum throughput. Exceeding this sweet spot will actually reduce aggregate throughput. To the extent that serverless applications are stateless and are unable to pool connections as well as a stateful application, you should expect to see the volume of connections to grow proportional to load, potentially knocking over the backing DB (or in Aurora Stateless's case, returning max connections errors).

Have you tried autoscaling on DynamoDB? Just curious how well that scales up. Clearly is not a drop-in replacement for MySQL.

It is best called "walletscaling." To run anything at scale you are looking at reserved capacity and provisioned throughput. Add extra fees for DynamoDB Accelerator (query caching), Global Tables (cross region replication), backups charged per GB, etc.

Most of the required addons aren't in the "cloud model" where you pay for usage, but instead you pay to have them on regardless of usage.

> DynamoDB Accelerator (query caching)

You don't need to use DynamoDB Accelerator though, and it provides value you won't automatically get for free by using another database. If you don't use it you will be managing your own redis/memcache instance with all cache invalidation logic.

> Global Tables (cross region replication),

Again, you don't get this for free in any other DB. Setting up your own multi master cross regional database is not free.

Dynamo DB works fine without any of the above two features.

>It is best called "walletscaling.

And which DB out of curiosity scales without any load on your wallet? What mythical DB can one run which needs neither horizontal nor vertical scaling, thus not impacting the wallet.

> Setting up your own multi master cross regional database is not free.

Is multi primary the new term? Has an agreed upon term been decided yet? Django seems to be switching to primary/replica.

https://docs.djangoproject.com/en/3.0/topics/db/multi-db/

> And which DB out of curiosity scales without any load on your wallet?

BigQuery includes all these things "for free" in the base price. It is pretty straightforward, with a price for storage and a price for querying (which you can choose to be usage or fixed). DynamoDB started off the same way, and added nickels and times by the roll.

You can't be serious when comparing BigQuery to DynamoDB. BigQuery is an OLAP database, you can't use it for OLTP workloads. And BigQuery has no equivalents to DAX(very low read latencies), Global Tables(millisecond access to data globally). If you are using BigQuery as an OLTP database with reads and writes going in simultaneously, more power to you.

DynamoDB costs you nothing when its not in use. It only charges based on actual reads and writes performed. As opposed to a traditional Relational database that always has to be on and has to be scaled up vertically to the maximum anticipated load permanently and with a read replica if you really want redundancy.

DynamoDB gives you redundancy out of the box (your tables are replicated across the three availability zones in a region), the scale is available to you if you have sudden traffic in on demand mode or you can set a limit if you wish to manage costs; your queries may receive errors about being throttled at some point if you approach those limits.

For OLTP workloads, DynamoDB (and a lot of other NoSQL-style, cluster based databases) cannot be beat for performance, capacity, scalability and costs. Which is exactly what you want on the front line of a workload that can receive large amounts of traffic.

For OLAP workloads with unknown query patterns across a variable set of data that can change over time and large table scans, a relational database is king because the actual volume of traffic is low but the size of queries are a lot larger usually.

Autoscaling is monstrously good on DynamoDB. You can turn on on demand mode so that it will scale up to the sheer limits of DynamoDB itself, but you pay per usage. So if you do want to set some kind of limit based on affordability you can set limits as to how much the tables scale.

As to how that autoscaling performs, its instant and always available so you have nothing to worry about there. Rather spend your time focussed on optimisation of queries than managing the scalability of your datastore which is as it should be.

Reply on news.ycombinator.com