YugaByteDB – A Transactional Database with Cassandra, Redis and PostgreSQL APIs | Hacker News Reader

YugaByteDB – A Transactional Database with Cassandra, Redis and PostgreSQL APIs

github.com

115 pointsmountainview31 comments

Does anyone know a comparison between this and CockroachDB? Or have experience running either in production?

They seem to compare against other databases, but not against Cockroach which seems to be the biggest competitor. I'm looking at implementing a global-scale database cluster with very specific requirements, and YugaByte seems to meet those but a comparison against CockroachDB seems warranted.

Edit: just noticed they do compare features against CockroachDB: https://docs.yugabyte.com/latest/comparisons/#distributed-sq..., but they don't have an in-depth comparison.

Postgres layer is still extremly beta (has no update support etc.)

But it should become good fast since they use fdw api (storage engine api in pg12) and reuse most of pg code.

Hi @lukeqsee, we are working on just this, please stay tuned!

YugaByte only added distributed transactions in the new 1.1 release (previously, it only supported single-row transactions). The architecture is a variation on 2PC and seems sound, but I think it's fair to say that it's early days. Meanwhile, Cockroach and TiDB both have battle-tested implementations.

YugaByte is pretty quirky. Rather than settle on a native data model, they offer several "personalities" that mimic other products (Cassandra, Redis and PostgreSQL), and you can mix and match them. But they've implemented each API with their own set of weird warts. For example, their CQL implementation has CREATE INDEX, but it does not index existing data [1]. You have to either create the secondary index before inserting data, or force a reindex of everything with a dummy UPDATE statement. Who would ship such a product?

More warts/Cassandraisms: UPDATE is actually an upsert; an update that doesn't match a row will insert it. And SELECT ... WHERE expressions can only use AND expressions (!) [2].

Hopefully the forthcoming SQL API should be saner, but it's very limited at the moment. It does not seem to support joins, transactions or indexes, for example. Meanwhile, CockroachDB and TiDB both have rich SQL implementations, including joins and aggregations, with cost-based query optimizers that can take advantage of multiple indexes and table statistics.

It seems more appropriate to compare YugaByte with distributed key/value stores like FoundationDB, Cassandra, Scylla and Redis.

[1] https://docs.yugabyte.com/latest/explore/transactional/secon...

[2] https://docs.yugabyte.com/v1.0/api/cassandra/dml_select/#roo...

Thanks atombender! This is helpful.

Those things you mentioned are just how Cassandra works. It's a wide-column key/value store and everything is an upsert because that's how the write path is designed, there is no read-then-write (other than LWTs). If you're using an application that expects Cassandra then this is normal so why would Yugabyte change the semantics?

It's true that indexing is unfinished but they've also long been problematic in the CQL data model so Yugabyte is moving faster and actually releasing something for those who can work with it. Cassandra took years to come up with several types of indexes that all have problems and Scylla is overdue by 18 months on their implementation.

That's why I referred to the behaviour as Cassandraisms.

The question was posed in the context of CockroachDB, so my answer stands -- CDB vs. YugaByte is an apples to oranges comparison, seeing as YugaByte isn't an RDBMS by any useful measure, and has a very long way to go to get there.

In the NoSQL space, FaunaDB indexes are very powerful. They are term partitioned and sharded and have compound terms, covered values, transformations, etc.

Cockroach treats the entire installation as one giant cluster, with optional 'locality' feature for each node to distribute data. Yugabyte has regional clusters, which can replicate from a single master write cluster. CRDB is working on ways to get fast local reads for different regions.

Other than that, Yugabyte also supports more protocols like Redis and Cassandra, and enhancements within them. It's really more of a distributed key/value store like FoundationDB but with protocol layers on top included out of the box.

When I see databases like this pop up, a part of me wonders why they don't devote effort to develop a plugin for MariaDB or PostgreSQL? Follow in the steps of Citus Data.

CTO of YugaByte here... this is indeed our plan. We are working with the community on pluggable storage coming out next year, and in the slightly longer term want to make YB a plugin (extension in the case of PG) model. While a reasonable amount of work is needed in order to get there, this is definitely the direction!

Here's a question -- what makes YugaByte unique? There's already an early but strong market here in the form of Google Spanner on GCP, Cockroach DB, and TiDB -- what does YugaByte bring that none of them do?

Multiple data models.

Just to be clear, this would mean a PG extension that works with the new PG pluggable storage to use Yugabyte as the backing store? Does that mean still running a separate Yugabyte cluster or is everything self-contained in the extension?

YugaByte DB's design is that a YB cluster supports Postgres in a native, self-contained & scale-out manner (much like YugaByte's Cassandra and Redis flavored offerings).

At a high-level, the upper half of the Postgres DB is being largely reused. The lower-half, i.e. the distributed table storage layer, uses YugaByte's underlying core-- a transactional and distributed document-based storage engine.

For the DB to be scalable, the lower-half being distributed is necessary but NOT sufficient. The upper-half also needs to be extended to be made aware of other nodes executing DDL/DML statements and dealing with related concurrency while still allowing for linear scale. Also, making the optimizer aware of the distributed nature of table storage is the other major piece of work in the upper-half.

These changes required in the upper half is what makes the "100% pure extension" model a bit harder... but that's something we intend to explore jointly with the Postgres community.

Hi @manigandham, great question. As it stands now, YugaByte would not be a PG extension. It will be a fully distributed but standalone PG cluster called "YSQL" API layer. We have started with the 10.4 version of PG for building distributed YSQL.

PG is planning the pluggable storage API for version 12.0, but these will only address user tables. This is certainly essential, but not sufficient for true distributed SQL. Some of the other pieces are: * Pluggable storage for system tables * Ability to create the initial set of system tables in a pluggable manner (initdb equivalent) * A number of other changes to the upper half of PG in order to make it "stateless" and scale-out, such as modifications to how certain operations are executed, which locks are held, what operations are pushed down to the underlying storage layer, etc. * Support for different types of indexes * Enhancing the optimizer to understand different storage types (in this case a distributed store)

In our current implementation, we have tried to make APIs for the above where ever possible. We need to explore how we can make runtime hooks for the other places. The plan is to see how to contribute these to the open source over the longer time frame so that we can create a self contained extension (though we are nowhere close to that today). Since our PG modifications code-base is open-sourced, the hope is that it would make contributing these changes back to PG easier.

Several months ago, setup of a Citus Cluster seemed to be a not straightforward, while at the same time I was able to setup and run a CockroachDB cluster in less than half hour with docker. I've checked YugaByte documentation about setup, and CockroachDB still won in operation complexity.

I doubt that the same will be done with PostgreSQL or MariaDB anytime soon. People are begging for a good and easy psql HA cluster setup for ages, and it is just not happening (yeah, they are getting closer and closer), while such DBs fill a need with good results.

@isoos, I'd be curious to understand more about why you thought YugaByte would be more operationally complex to deploy, as we strive to make things as easy to get started with as possible...

For example, for local testing, in our Quick Start docs section, we have info for: mac/linux, involving just downloading and unpacking the release and you should be good to go; for local docker, you can download our control script; for k8s download our sample yml.

Finally, for non-local testing, we have a Deploy > Manual Deployment section, highlighting the 3 steps for downloading YugaByte, bringing up Masters (metadata nodes) and then Tservers (data nodes).

Note: I work at YugaByte.

With CockroachDB, I was able to create a cluster with three commands, one command on each node (docker run ....), one port on each node. It can't really get more simple than that.

With YugaByteDB: I needed to think about master and tserver nodes, to run them and connect them separately. There was no clear guide that had "run these three docker command on the three different nodes" and be done with it.

Note: I run everything is in simple docker images (and not compose, not swarm, not kubernates, just plain docker images).

CRDB is only has one node type which are all symmetrical, deployed as a single binary. It doesn't get simpler than that.

Most other databases insist on multiple layers and dependencies which only shifts more work onto developers who have to install and maintain it.

When I tried Citus, their documentation has everything except what I needed to do to get their product to work. My guess is they want to sell you engineering support. The docs might have been updated, but I do agree with the docs.

> People are begging for a good and easy psql HA cluster setup for ages

I've been using Stolon in production for a bit, and while not a native solution- it works quite well.

Stolon has no sharding

It has HA. Sharding is a different beast which I think Citus has solved.

Can you combine it with some other tool that has sharding ?

I think Citus can do both- sharing and replication. The docs are not 100% clear on this, but they do talk about replication and sharing together.

To use a solution like Stolon would require changes in the core logic to dynamically re-configure whatever sharing plugin you use.

I can't speak to PostgreSQL, but I work on the TiDB team having previously worked on the MySQL team.

Ensuring compatibility from a clean slate is hard, but to make MySQL distributed requires more than the current storage engine API provides. Thus, you would be likely be creating a fork and no longer getting the benefits of having an upstream[1].

The surgery you need to perform on the code would also reduce you from full compatibility (switching from pessimistic to optimistic locking means some statements no longer work). There are some great side benefits to a clean slate too: In TiDB we're able to use modern languages such as Go for the TiDB Server, and Rust for TiKV.

([1] Phrasing this question to upstream: I think they'd like to offer such functionality, but introducing this level of backward incompatibility is hard. They are also very concerned about performance regression bugs, which would be very hard to prevent with all the refactoring required.)

Citus is a great product, but it's still fundamentally hamstrung by the foundations of the underlying database.

CockroachDB gives you proper data replication and high-availability at a high density. Postgres doesnt, and requires an entirely separate cluster with fragile replication and manual cutover.

Those are a lot of claims. Databases that make half those claim, turn out to be misrepresenting some of them. So far its empty promises on a box.

When can we expect independent 3rd party evaluation of all these claims? The most famous one is Jespen:

https://jepsen.io/analyses

Because this all sounds way too good to be true. What's the catch? How stable is this? Are the claims tested other than in theory?

They made a post about testing with Jepsen: https://blog.yugabyte.com/jepsen-testing-on-yugabyte-db-data...

Awesome. Thanks. This is very very encouraging!

Design is very similar to Hbase minus the dependency on Zookeeper and HDFS. RocksDB has been very smartly modified to use it as an optimized storage layer.

Reply on news.ycombinator.com