derekperkins

Vitess still supports foreign keys (single shard), using MySQL, just not with the Online DDL functionality.

I think Cockroach tries to be a little too magical, which is great for starting up a cluster, but I think you can architect for much better performance with Vitess and owning your sharding model. I'm also very happy to use InnoDB, one of the most battle tested db engines to ever exist, while Cockroach is currently rewriting theirs from scratch. At the distributed level, I don't know of any massive scale adopters of Cockroach yet, though I'm not 100% looped in, so forgive me if I'm ignorant of them. On the other hand, Vitess has seen adopters like Slack, GitHub, Square, HubSpot, YouTube, with many more in various stages of adoption.

I feel like Citus might be trying to be too many things and so hasn't gotten the traction that Vitess has. Vitess has nailed OLTP at scale, while Citus is trying to also do OLAP and be a single source. That's the holy grail, but I'm not sure that any technology is close to handling both of those well yet.

I do use it in production and have for years, just not the online schema changes. It's fantastic and FKs are supported in a single shard, which we use heavily.

I'm a Vitess maintainer and I feel the same way. I don't plan to use any of the Online DDL because you'll have to pry my foreign keys out of my cold, dead hands. I understand the reasoning and limitations, but like you, the trade-off isn't worth it to me.

That's my favorite part about using messaging in Vitess. I can emit events in a real transaction. It eliminates an entire class of bugs.

https://vitess.io/docs/reference/features/messaging/

Solving both these problems is the best hidden feature of Vitess - Messaging. You can ack a message, do data work, and add to a destination queue all in a single transaction. You can do selective requeuing, and infinite parallelization, since it isn't sequential processing. It's easy to introspect, debug, and have metrics for since it's just MySQL. Vitess lets you shard horizontally, so it can handle any QPS, and has at YouTube. It also supports native message priority. All of that, plus your infrastructure is simplified because you don't have to maintain a separate data store from your main RDBMS. Highly recommended. https://vitess.io/docs/reference/features/messaging/

I believe most of those are self-hosted Vitess with Planetscale providing technical resources and support.

In practice this doesn't turn out to be a major issue, since a good design will keep >99% of queries/transactions on a single shard.

4 years of Vitess in prod, also a maintainer

That's pretty close to how Vitess handles sequences (distributed auto-increment)

Insane overhead in what way?

It's even more hypocritical than just cherry picking studies. Points in his books are refuted by the same studies he cites! He ignores similar / better outcomes that don't match his bias.

We tried the same thing a few years ago in MySQL that was randomly non-performant: 4000x slower. Still waiting for that to be fixed.

They'll also help you host Vitess, not sure if it's fully managed or not, but that'd get it in your cluster / region.

Have you seen any of those since 5.7?

Still do

Yeah, Google policy was to migrate everything to Spanner, not a technical decision.

> Are cross-server transactions scalable? Would anyone architect a system that did them en masse unless they absolutely 100% had to?

As a long time Vitess user and contributor/maintainer, this perfectly describes the issue. Minimal effort has been put into solving cross shard / 2PC transactions because a well designed system rarely needs it.

Vitess does just that, letting you seamlessly scale writes in the same way you would scale reads today, scaling linearly with additional nodes.

Most of these are Postgres specific, though MySQL is going to have its own list of issues.

That's why Vitess is so awesome - you can scale writes infinitely. There's not a truly comparable option for Postgres

We just use the git sha and they works great, no reason to overcomplicate it

I think the vast majority of devs work in a single language, at least on a day to day basis.

That was added a couple years ago in MySQL 8, as were window functions and arbitrary checks. The two databases continue to fill in the gaps that were historically reasons to choose one over the other.

Those can both be simultaneously true. Employers can be supportive while still trying to regain lost productivity.

5TB is not that large and it's not that difficult to get good performance. We operated a 50TB single instance of MySQL for 5 years with a tiny team before migrating to Vitess, and it was basically zero maintenance. We did partition our largest table, which just requires a little extra SQL, but is otherwise transparent.

There is definitely a magical feeling to spinning up Cockroach on a single binary and just letting it do things for you, vs the much steeper learning curve of Vitess. If you aren't looking for peak performance and are optimizing for rapid iteration and ease of use, that's a more compelling pitch than I've heard before. I've always recommended people choose either Postgres or MySQL, whichever they feel more comfortable running, since they likely won't have to ever migrate off the single db, and they'll have full compatibility for queries, apps, etc.

> Those companies exist, sure, but it's not most of the Fortune 500

That's a very interesting observation on database scale and not one that I've heard before, but I guess makes counter-intuitive sense. Most of the Fortune 500 aren't modern tech companies, and thus may not be as reliant on databases in the same way as a newer company.

It's not that it can't be, but it is a completely different codebase from Spanner without the luxury of TrueTime and complete control over datacenters that it runs in. Cockroach simply hasn't been pushed to the same levels of performance that Spanner or Vitess are already serving, and likely a few orders of magnitude less. I look forward to seeing case studies when larger enterprises do take it that far.

Since I'm not a Cockroach expert, I'm sure that there are ways to configure it that eliminate some of those latency issues that were common early on. Both Vitess & Cockroach can replicate across regions and survive regional failures.

As for cross-shard transactions, that's mainly what I'm referring to about slowness. Vitess does have 2PC, but as you mention, almost nobody uses it in production. Companies using Vitess architect their data and sharding patterns so that cross-shard transactions aren't necessary. There are definitely cases where that is part of the requirement, but in far fewer cases than many people think.

At the end of the day, I'm still waiting to see any major tech company push serious workloads onto Cockroach. Until that happens at scale, and as I continue to see more and more of those major companies adopting Vitess, it's going to be hard to convince me that Cockroach solves all the problems that they claim to. What they are fantastic at is great docs and beautiful interfaces, both of which are lacking on the Vitess side. :)

I don't really understand the continuing appeal for CockroachDB. Yes there is some cool tech, but given that all transactions are synchronously distributed, there's a hard cap on how fast it can be. I have yet to see any significant case studies about real workloads, while their competitors like Vitess are seeing whole-scale adoption from their users - including Slack and HubSpot.

Whatever the service, it isn't the platform that Slack is. I would guess that a huge percentage of their db traffic is from apps and custom integrations.

That introduces different problems. Applying MySQL replication logs is single threaded by default and generally doesn't scale as well as the master, so it becomes increasingly difficult to keep replicas from lagging.

As your database gets into tens or hundreds of TBs, now backups and restores take hours or days to complete. Any DR scenario becomes an existential threat.

For large tables, schema changes can take weeks. This often results in developers doing their own custom table sharding strategies, even though they might still be on a single machine.

I'm not saying that you can't architect around these problems, but operating a db at massive scale on a single machine isn't the obvious win that it seems to be.

They are providing a managed Vitess service, which allows you to use almost any software that is compatible with MySQL.