Honest question, why not just use dynamodb? If I'm already mostly server less and AWS, dynamodb seems like a natural choice for a persistent store in a world where cost and zero-scaling is important.
> Some unsuitable workloads for DynamoDB include:
> Services that require ad hoc query access. Though it’s possible to use external relational frameworks to implement entity relationships across DynamoDB tables, these are generally cumbersome.
> Online analytical processing (OLAP)/data warehouse implementations. These types of applications generally require distribution and the joining of fact and dimension tables that inherently provide a normalized (relational) view of your data.
> Binary large object (BLOB) storage. DynamoDB can store binary items up to 400 KB, but DynamoDB is not generally suited to storing documents or images. A better architectural pattern for this implementation is to store pointers to Amazon S3 objects in a DynamoDB table.
For each of these workloads you switch the query pattern to accommodate for DynamoDB and provide a solution for your workload and/or app. That is actually the secret of NoSQL. You do the work upfront.
So your reading of the recommendations is correct on the surface but incorrect on the fundamental usage aspect of it. Lets have a look at each:
> Services that require ad hoc query access. Though it’s possible to use external relational frameworks to implement entity relationships across DynamoDB tables, these are generally cumbersome.
A: Dont do this. Don't do ad hoc query access. Define a series of access patterns and lay out your data to support them. You dont get ad hoc query access for Netflix, Amazon or your Airline Travel website...
> Online analytical processing (OLAP)/data warehouse implementations. These types of applications generally require distribution and the joining of fact and dimension tables that inherently provide a normalized (relational) view of your data.
These are not most SQL Patterns. This is OLAP and was always or should be done with MPP systems, not with your relational database like your SQLServer or your Oracle. So this is going on an edge...
> Binary large object (BLOB) storage. DynamoDB can store binary items up to 400 KB, but DynamoDB is not generally suited to storing documents or images. A better architectural pattern for this implementation is to store pointers to Amazon S3 objects in a DynamoDB table.
Your Relational Database will not store BLOBs more efficiently than S3 anyway....
"When is "ACID" ACID? Rarely." - http://www.bailis.org/blog/when-is-acid-acid-rarely/
So your post is both irrelevant as a response to mine about consistency and more generally irrelevant to the entire discussion here.
From the article: "The textbook definition of ACID Isolation is serializability (e.g., Architecture of a Database System, Section 6.2), which states that the outcome of executing a set of transactions should be equivalent to some serial execution of those transactions. This means that each transaction gets to operate on the database as if it were running by itself, which ensures database correctness, or consistency."
" This means that each transaction gets to operate on the database as if it were running by itself, which ensures database correctness, or consistency. A database with serializability (“I” in ACID), provides arbitrary read/write transactions and guarantees consistency (“C” in ACID), or correctness, of the database. Without serializability, ACID, particularly consistency, is generally not guaranteed"
I am of course ignoring the Consistency you are certainly aware that exist in DynamoDB with Strong Consistency and DynamoDB Transactions.
Moving consistency management outside of DynamoDB as you propose doesn't circumvent the CAP theorem's limitations.
How would you do it? Assume we want proper pagination, and not rewrite the app for cursor based "Load more" style pagination. Why? Because the React Admin provider API insists. https://github.com/marmelab/react-admin/issues/1510
You are not supposed to do the same query patterns. My argument is that you can substitute your relational database, by changing the app and the layout of the data to match the proper patterns for DynamoDB.
"Migrating to DynamoDB from a relational database" - https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
""How to model one-to-many relationships in DynamoDB" - https://www.alexdebrie.com/posts/dynamodb-one-to-many/
You can go further by using the streams feature to dump your data into an analytical database for your querying needs.
It gets worse than that though, as Dynamo's usage on large tables depends on its hottest shard, and other than setting primary keys, you have no control over how it shards. The story of people moving to dynamo and touting its advantages, just to move out a year later because the shape of their data makes dynamo prohibitively expensive as the data grows is pretty common. I once worked at a place where they stored historical data in dynamo, leading to a substandard hash key. Most of the time the db was idle, but when it wasn't, the sharding scheme made them pay a good 500x capacity than it was actually in use, because while a shard was red hot, others were completely idle. The monthly price for this relatively unimportant feature ended up being higher than the office's rent, in San Francisco.
If your table's key is UUIDs, and the chances of querying one record or another is almost perfectly flat, then sure, dynamo away! But if you walk away from it very best use case, and you suddenly start having any amount of data... dynamo can become really, really expensive. I'd not say you should never use it, but it's a really scary first place to go, precisely if you care about costs.