Show HN: I built a MySQL storage engine which is InnoDB compatible

What concerned me a bit was absence of an interesting and promising general idea behind the implementation, the reason to implement a new engine. Maybe it is there, but the blog post does not say anything about it.

MVCC is so popular because it makes ACID compliance much easier to implement, but there is some additional read latency (and management overhead) because storage layout is disconnected from the natural layout.

A storage engine which sticks to the basics could be very fast, with predictable low-latency.

With regards to threading. I know locks are a dirty word, but you do need a single version of truth somewhere. Either this is done through locking or an allocator. Going single-threaded is a valid way to remove the overhead of locking (plus no race conditions!).

Hopefully a single writer, multiple readers is available soon.

The reason is that most of the performance is spent in MySQL and not in the key/value store, and then it does not make a big difference if the key/value store is concurrent or not.

In my experience the assumption of "concurrent = fast" is a misconception. Right now upscaledb moves certain operations (i.e. flushing dirty buffers) to the background. It is better to have fast single-threaded code instead of multi-threaded code with a huge locking overhead. A compromise would be to move the lock to the database level (instead of the Environment, which is basically the container for multiple databases), and make sure that there's no shared state between the databases. But that actually does not have that much priority for me because I do not expect to win that much performance.

upscaledb does use transactions. I.e. if you insert a row with a primary and a secondary index then upscaledb inserts two key/value pairs in two databases. All these operations are wrapped in Transactions. They are just not yet supported on SQL level.

Ironically, back in the day, the core MySQL team argued that was a feature, not a bug.