> Please keep reading to see how diligent we were in creating a fair test case
I hope this is true, but my cursory reading found several unfair spots that seem at odds with this statement.
> 3-node cluster on single DC | RF=3
DynamoDB works across AZs and will continue to be available even when a data center goes down. That cross-AZ operation has benefits as well as latency costs that are not present in the ScyllaDB setup.
> We hit errors on ~50% of the YCSB threads causing them to die when using ≥50% of write provisioned capacity
That is surprising. It is quite common to use your full provisioned capacity without problems. My guess is that there is something not ideal about the YCSB DynamoDB library or its configuration. I'm not familiar with YCSB: does it give you stack traces that indicate why the threads failed?
> Sadly for DynamoDB, each item weighted 1.1kb – YCSB default schema, thus each write originated in two accesses
This is what made me come write this comment. You specifically knew this was a pessimistic case which could be easily addressed to allow DynamoDB to operate at a lower cost. Is arbitrarily settling for the default 1.1kB item size on an artificial benchmark fair? Good engineering teams use their tools the way that gives them the most benefit. Calling out that you may have to work to ensure your use case doesn't have pessimistic characteristics would clearly be fair, but I'm not convinced that just picking arbitrary benchmark settings is.